Re: [Resin-interest] Scaling Up
On Nov 10, 2012, at 7:14 AM, Haroon Rasheed wrote: Thanks Paul for the great Video! Could you please provide more details on the clustered sessions using distributed memcache/JCache? Hi Haroon, The Memcache/JCache layer really isn't intended for direct access to the user session, if that was the intention of your question. Although theoretically that's possible. We've added these API so that your application can setup and use a distributed cache as you see fit. This cache is based on the same technology Resin uses for session failover. There is a memcached tier example in resin.xml in the latest Resin releases. It sets up a Memcache network listener. Here are some links to documentation on using the distributed cache with JCache: http://www.infoq.com/news/2011/12/resin-memcached http://www.caucho.com/resin-application-server/press/caucho-features-resin-cache-an-elastic-and-persisted-alternative-memcached-solution/ http://caucho.com/resin-4.0/admin/cluster-cloud.xtp#ClusterResourcesCacheQueues http://wiki4.caucho.com/Resin_Java_Cache_using_CDI_Inject Thanks, Paul Regards, Haroon On Sat, Nov 10, 2012 at 3:41 AM, Paul Cowan co...@caucho.com wrote: On Nov 8, 2012, at 10:25 PM, Rick Mann wrote: On Nov 8, 2012, at 19:08 , Scott Ferguson f...@caucho.com wrote: We're working on a video that should help. One more request: I really prefer text descriptions. I find videos excruciatingly difficult to sit through, whereas a well-written textual how-to allows me to skim over parts I'm familiar with. Oh no don't say that! I just finished a cluster video specifically because people were asking the same questions you brought up. http://tinyurl.com/aprccda I would be happy to hear any feedback you or others have before the final cut. Thanks, Paul One of the keys to making it well-written is to put the actual steps to carry out in a bullet-point or numbered list, so that a crucial bit of info isn't buried in a paragraph somewhere. Thanks! -- Rick ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest === Paul Cowan, Software Engineer Caucho Technology co...@caucho.com http://blog.caucho.com http://twitter.com/cauchoresin ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest === Paul Cowan, Software Engineer Caucho Technology co...@caucho.com http://blog.caucho.com http://twitter.com/cauchoresin ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Scaling Up
On Nov 10, 2012, at 5:56 AM, Riccardo Cohen wrote: Really great video. Looks magic ! I've been setting load balancing with resin.xml only. I cannot find resin.properties and resinctl script in my resin 4.0.13 installation. When did it appear ? did you announce somewhere this new function (maybe I don't receive caucho's customer's news ? or maybe I was not watchful enough...) Here's a link to a handout we created for JavaOne this year. The back (2nd page) summarizes all the recent enhancements. It should be helpful: https://s3.amazonaws.com/caucho-web/Resin_4.x_Cloud_JAMM_overview.pdf Typically the best place to look for changes is the release notes: http://caucho.com/resin-4.0/changes/index.xtp resin.properties was added in version 4.0.24 and refined in .25. Documentation here: http://caucho.com/resin-4.0/admin/config-resin-properties.xtp Thanks, Paul === Paul Cowan, Software Engineer Caucho Technology co...@caucho.com http://blog.caucho.com http://twitter.com/cauchoresin ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Scaling Up
you'll find my 2 xml conf (resin 4.0.13)
- resin.xml is the front server, I have 1 front for 2 app-tier clusters,
one cluster is for the application, the other if for images only servers
- resin.safe.xml is the conf for application app-tier, I did not set any
http service for this because I never access it directly. (but you have
to if you use resin-admin)
I don't use automatic deployement or elastic server yet, but I think
I'll try this on the new version of resin.
good luck
On 09/11/12 09:52, Rick Mann wrote:
On Nov 9, 2012, at 0:19 , Riccardo Cohen r.co...@realty-property.com wrote:
Hello Rick
I haven't worked a lot on this, but if you want I can provide my
resin.xml : in my application I have one resin front loadbalancer, with
N application servers (all IP are known) and potentially 2 mysql db
servers .
I use resin 4.0.13 pro on linux debian
Thanks, Riccardo, couldn't hurt!
--
Riccardo Cohen
+33 (0)6 09 83 64 49
Société Realty-Property.com
1 rue de la Monnaie
37000 Tours
France
http://www.appartement-maison.fr
!--
- Resin 4.0 configuration file for a clustered/load-balance environment
--
resin xmlns=http://caucho.com/ns/resin;
xmlns:resin=urn:java:com.caucho.resin
!-- adds all .jar files under the resin/lib directory --
class-loader
tree-loader path=${resin.home}/ext-lib/
tree-loader path=${resin.root}/ext-lib/
tree-loader path=${resin.home}/lib/
tree-loader path=${resin.root}/lib/
/class-loader
!--
- jars with maven dependencies can be placed in project-jars to be used
- by web-apps
resin:ProjectJarRepository path=${resin.root}/project-jars/
--
!--
- Resin-managed data, including sessions, administration, and clustering
--
resin-data-directory${resin.root}/resin-data/resin-data-directory
!--
- Remote management requires at least one enabled admin user.
resin:AdminAuthenticator
resin:import path=${resin.root}/conf/admin-users.xml optional=true/
/resin:AdminAuthenticator
--
!--
- Logging configuration for the JDK logging API.
--
log-handler name= level=info path=/home/webapps.tmp/realty/logs/jvm.log
timestamp=[%y-%m-%d %H:%M:%S.%s] {%{thread}}
rollover-size=1mb
rollover-period=1D
rollover-count=60/
!--
- 'info' for production
- 'fine' or 'finer' for development and troubleshooting
--
logger name=com.caucho level=info/
logger name=com.caucho.java level=config/
logger name=com.caucho.loader level=config/
logger name=com.caucho.sql level=info/
logger name=com.caucho.amber level=info/
logger name=com.caucho.server.http level=info/
stdout-log path='/home/webapps.tmp/realty/logs/jvm.log' timestamp=OUT[%y-%m-%d %H:%M:%S.%s] {%{thread}} /
stderr-log path='/home/webapps.tmp/realty/logs/jvm.log' timestamp=ERR[%y-%m-%d %H:%M:%S.%s] {%{thread}} /
!--
- For production sites, change dependency-check-interval to something
- like 600s, so it only checks for updates every 10 minutes.
--
dependency-check-interval-1s/dependency-check-interval
!--
- SMTP server for sending mail notifications
--
system-property mail.smtp.host=smtp.nothing.fr/
system-property mail.smtp.port=25/
!--
- Sets the default character encoding to utf-8
-
--
character-encodingutf-8/character-encoding
!--
- You can change the compiler to javac, eclipse or internal.
--
javac compiler=javac args=/
cluster-default
!-- standard servlet behavior, including .jsp, .php, and WEB-INF --
resin:import path=${resin.root}/conf/app-default.xml/
!--
- Admin services
--
resin:DeployService/
resin:if test=${resin.professional}
resin:JmxService/
resin:LogService/
resin:StatService/
resin:XaLogService/
/resin:if
/cluster-default
cluster id=realty
!-- sets the content root for the cluster, relative to resin.root --
root-directory./root-directory
!-- enables development error pages
development-mode-error-page/
--
server-default
!-- The http port
http address=* port=80/
--
!-- SSL port configuration:
http address=* port=8443
openssl
certificate-filekeys/gryffindor.crt/certificate-file
certificate-key-filekeys/gryffindor.key/certificate-key-file
passwordmy-password/password
/openssl
/http
--
!--
- The JVM arguments
--
jvm-arg-Xms64m/jvm-arg
jvm-arg-Xmx256m/jvm-arg
jvm-arg-Xss1m/jvm-arg
jvm-arg-Dfile.encoding=UTF-8/jvm-arg
!--
jvm-arg-Xdebug/jvm-arg
jvm-arg-Dcom.sun.management.jmxremote/jvm-arg
--
!--
- Uncomment to enable admin heap dumps
- jvm-arg-agentlib:resin/jvm-arg
-
- Uncomment for low-memory overhead
- jvm-arg-Dcaucho.smallmem/jvm-arg
--
!--
- arguments for the watchdog process
Re: [Resin-interest] Scaling Up
Hello Rick I haven't worked a lot on this, but if you want I can provide my resin.xml : in my application I have one resin front loadbalancer, with N application servers (all IP are known) and potentially 2 mysql db servers . I use resin 4.0.13 pro on linux debian On 09/11/12 03:11, Rick Mann wrote: I've used Resin for 6 or 8 years now, ever since working at a company that used it. I use it for a lot of little personal projects and web sites, nothing very substantial. But there's a chance that something I did recently will grow, and I'm looking at how to use Resin with Rackspace virtual hosts. I'm finding it hard to get through some of Caucho's concepts. How is a web server different from an app server? Is a web server the same as a load balancer? All my apps run completely self-contained on a single server, talking to a MySQL instance running on the same machine, using Hibernate and no special caching. I'm having a hard time quickly learning how to move from that environment to the triad and elastic servers. Can I take the intermediate step of running just two servers? How does load balancing work? I've only ever used a Big IP F5. Do I want to bother with Rackspace's load balancers? Stuff like that. Is there a good introductory document (not just a set of slides)? So far, everything I've read assumes a certain level of knowledge. Thanks, -- Riccardo Cohen +33 (0)6 09 83 64 49 Société Realty-Property.com 1 rue de la Monnaie 37000 Tours France http://www.appartement-maison.fr ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Scaling Up
On Nov 9, 2012, at 0:19 , Riccardo Cohen r.co...@realty-property.com wrote: Hello Rick I haven't worked a lot on this, but if you want I can provide my resin.xml : in my application I have one resin front loadbalancer, with N application servers (all IP are known) and potentially 2 mysql db servers . I use resin 4.0.13 pro on linux debian Thanks, Riccardo, couldn't hurt! -- Rick ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Scaling Up
On Nov 9, 2012, at 18:19 , Scott Ferguson f...@caucho.com wrote: - My current app has no session state at all, but does intantiate a lot of Hibernate objects in the app. Hibernate caching adds to this (I don't know how writes are handled). Do you have recommendations for ensuring failover works properly in this situation? Since I'm not a hibernate expert, someone else might need to answer this. I might actually be wrong about this one. Without explicit (Hibernate) caching, I think it's actually creating a persistence session on each request, and so all objects are created and destroyed in the request. Hibernate caching is its own beast, I'm sure I'll have to tackle that next. -- Rick ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Scaling Up
On Nov 9, 2012, at 18:19 , Scott Ferguson f...@caucho.com wrote: There's failover to backup triad members if the primary fails. There's an ack from the triad to tell when it's been completed. That's the basic idea. The cache is actually a fairly complicated messaging application. It's been lots of fun to implement. Does that ack happen before the session.setAttribute() call returns? If not, it seems like there might be a hole. -- Rick ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Scaling Up
On 11/8/12 6:11 PM, Rick Mann wrote: I've used Resin for 6 or 8 years now, ever since working at a company that used it. I use it for a lot of little personal projects and web sites, nothing very substantial. But there's a chance that something I did recently will grow, and I'm looking at how to use Resin with Rackspace virtual hosts. I'm finding it hard to get through some of Caucho's concepts. How is a web server different from an app server? Is a web server the same as a load balancer? All my apps run completely self-contained on a single server, talking to a MySQL instance running on the same machine, using Hibernate and no special caching. I'm having a hard time quickly learning how to move from that environment to the triad and elastic servers. Can I take the intermediate step of running just two servers? How does load balancing work? I've only ever used a Big IP F5. Do I want to bother with Rackspace's load balancers? Stuff like that. Is there a good introductory document (not just a set of slides)? So far, everything I've read assumes a certain level of knowledge. We're working on a video that should help. In the meantime: 1) Basically, we start with a single server, just like you're using now. That's an app server because it runs your applications (it also serves static web requests of course.). app_servers : 127.0.0.1:6800 2) Now you either want or need a second app server, either because of load or memory or reliability. That would be two servers. The two servers would be the first two in the triad. The triad stuff happens automatically. The first servers in your list are triad members. In truth, you don't usually need to know about the triad, except you should take a little care to put them on different hardware for reliability. app_servers : 192.168.0.10:6800, 192.168.0.11:6800 (You can also put two Resin servers on one box. Just use the same IP and different ports, like 6801. We use this for testing all the time, and some people do that for reliability or better memory management on one machine.) 3) load balancing But now there's a difficulty because some how the requests need to be shared across the multiple servers, so you need a load balancer of some sort. Resin doesn't care what load balancer you use. We're trying to support whatever configuration you want, assuming that you have a better idea of how you want to balance your own load. Resin's load balancer is mostly provided as an option for medium sites so you don't need to use a hardware load balancer, and because it's easy to configure. (Some sites even use a hardware load balancer in combination with Resin's load balancer.) If you're using a non-Resin load balancer, just point it to the 2 server's http ports. If you're using Resin's load balancer, just add a web_server entry: web_servers : 192.168.1.10:6820 In this case, I put the load balancer on the first machine. That's a perfectly reasonable configuration for a smallish configuration. Make sure all servers are using the same resin.xml and resin.properties. The entire point of Resin's design is to allow you to use a single configuration for your entire cluster. 4) Adding servers. You can keep adding servers to the app_servers list. The first three are automatically the triad. You don't need to do anything special, but if you have different physical servers, it's a good idea to split them for reliability. As long as every server sees that app_servers, the load balancing will just work with the new servers. 5) web vs app tiers. The idea of those two is to mirror the common Apache/Resin or nginx/Resin reverse proxy configuration where you replace Apache with another Resin. You can use that configuration even if you have one app-server. (Benefit is that it isolates the application from the actual network. The penalty is a bit of performance.) The web tier is simple and more reliable because it's only serving HTTP requests and cached data. It also isolates the web from the app tier, because only the web tier needs to worry about things like flaky network connections. When you're using Resin as a load balancer the web tier is the load balancer. 6) The triad. Mostly, it's an internal Resin clustering concept. The main purpose is to handle clustering for fairly large sites that have large variations in load where they want to add or drop servers frequently. The triad servers are the reliable servers. You don't actually need 3. If you have 2, the 2 servers act as backups for each other. The key of the triad is that every other server: 4, 5, 6 can be can be added or dropped whenever you want without affecting things like clustered sessions or caching. For example, you can configure 7 servers, and only have 3 live. Shut down the last 4 and the system just works. But _don't_ shut down the first 3, because that's the triad. That's really the only reason you need to know about the
Re: [Resin-interest] Scaling Up
Thanks, Scott, I'll read through that more carefully shortly. One question. Do I set up one server with the multi-machine additions as you show, and then just replicate the config file (and server, webapp, et al) onto all the machines? I realize that may seem obvious, but I want to be sure. Also, how do I get them to talk to each other via private interfaces? Or does that happen magically with Rackspace? -- Rick On Nov 8, 2012, at 19:08 , Scott Ferguson f...@caucho.com wrote: On 11/8/12 6:11 PM, Rick Mann wrote: I've used Resin for 6 or 8 years now, ever since working at a company that used it. I use it for a lot of little personal projects and web sites, nothing very substantial. But there's a chance that something I did recently will grow, and I'm looking at how to use Resin with Rackspace virtual hosts. I'm finding it hard to get through some of Caucho's concepts. How is a web server different from an app server? Is a web server the same as a load balancer? All my apps run completely self-contained on a single server, talking to a MySQL instance running on the same machine, using Hibernate and no special caching. I'm having a hard time quickly learning how to move from that environment to the triad and elastic servers. Can I take the intermediate step of running just two servers? How does load balancing work? I've only ever used a Big IP F5. Do I want to bother with Rackspace's load balancers? Stuff like that. Is there a good introductory document (not just a set of slides)? So far, everything I've read assumes a certain level of knowledge. We're working on a video that should help. In the meantime: 1) Basically, we start with a single server, just like you're using now. That's an app server because it runs your applications (it also serves static web requests of course.). app_servers : 127.0.0.1:6800 2) Now you either want or need a second app server, either because of load or memory or reliability. That would be two servers. The two servers would be the first two in the triad. The triad stuff happens automatically. The first servers in your list are triad members. In truth, you don't usually need to know about the triad, except you should take a little care to put them on different hardware for reliability. app_servers : 192.168.0.10:6800, 192.168.0.11:6800 (You can also put two Resin servers on one box. Just use the same IP and different ports, like 6801. We use this for testing all the time, and some people do that for reliability or better memory management on one machine.) 3) load balancing But now there's a difficulty because some how the requests need to be shared across the multiple servers, so you need a load balancer of some sort. Resin doesn't care what load balancer you use. We're trying to support whatever configuration you want, assuming that you have a better idea of how you want to balance your own load. Resin's load balancer is mostly provided as an option for medium sites so you don't need to use a hardware load balancer, and because it's easy to configure. (Some sites even use a hardware load balancer in combination with Resin's load balancer.) If you're using a non-Resin load balancer, just point it to the 2 server's http ports. If you're using Resin's load balancer, just add a web_server entry: web_servers : 192.168.1.10:6820 In this case, I put the load balancer on the first machine. That's a perfectly reasonable configuration for a smallish configuration. Make sure all servers are using the same resin.xml and resin.properties. The entire point of Resin's design is to allow you to use a single configuration for your entire cluster. 4) Adding servers. You can keep adding servers to the app_servers list. The first three are automatically the triad. You don't need to do anything special, but if you have different physical servers, it's a good idea to split them for reliability. As long as every server sees that app_servers, the load balancing will just work with the new servers. 5) web vs app tiers. The idea of those two is to mirror the common Apache/Resin or nginx/Resin reverse proxy configuration where you replace Apache with another Resin. You can use that configuration even if you have one app-server. (Benefit is that it isolates the application from the actual network. The penalty is a bit of performance.) The web tier is simple and more reliable because it's only serving HTTP requests and cached data. It also isolates the web from the app tier, because only the web tier needs to worry about things like flaky network connections. When you're using Resin as a load balancer the web tier is the load balancer. 6) The triad. Mostly, it's an internal Resin clustering concept. The main purpose is to handle clustering for fairly large sites that have large variations in load where they want
Re: [Resin-interest] Scaling Up
On Nov 8, 2012, at 19:08 , Scott Ferguson f...@caucho.com wrote: We're working on a video that should help. One more request: I really prefer text descriptions. I find videos excruciatingly difficult to sit through, whereas a well-written textual how-to allows me to skim over parts I'm familiar with. One of the keys to making it well-written is to put the actual steps to carry out in a bullet-point or numbered list, so that a crucial bit of info isn't buried in a paragraph somewhere. Thanks! -- Rick ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Scaling Up
On 11/8/12 7:17 PM, Rick Mann wrote: Thanks, Scott, I'll read through that more carefully shortly. One question. Do I set up one server with the multi-machine additions as you show, and then just replicate the config file (and server, webapp, et al) onto all the machines? I realize that may seem obvious, but I want to be sure. Yes, all the servers will have the same configuration. Also, how do I get them to talk to each other via private interfaces? Or does that happen magically with Rackspace? Ah. I wasn't explicit about that. (You're right about assumptions.) The IP addresses you put in app_servers and web_servers are *private* addresses. You shouldn't put public addresses in the app_servers and web_servers, if you have the option. If you have 1 machine, you should use 127.0.0.1, otherwise use the private IP. I don't specifically know about Rackspace, but you might need to change the firewall settings so each server is allowed to connect to 6800. Amazon, for example, has a security group firewall, so only the servers in the security group are allowed to see 6800. Outside servers can't get to it. I'm sure Rackspace has something similar. When you start Resin with start-all (or the /etc/init.d script), Resin will compare its own IP address with the configuration IP addresses in app_servers and web_servers and start itself as the local address. (Does that make sense?) -- Scott ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Scaling Up
On Nov 8, 2012, at 19:28 , Scott Ferguson f...@caucho.com wrote: Yes, all the servers will have the same configuration. Okay, so, I can set one up with the config, make a snapshot of that machine, then bring up more instances from that snapshot. I'll have to figure out how to manage the IP addresses of the hosts that I bring up; might have to manually edit the config file after they're brought up. Similar question for elastic servers. Also, how do I get them to talk to each other via private interfaces? Or does that happen magically with Rackspace? Ah. I wasn't explicit about that. (You're right about assumptions.) :-) The IP addresses you put in app_servers and web_servers are *private* addresses. You shouldn't put public addresses in the app_servers and web_servers, if you have the option. If you have 1 machine, you should use 127.0.0.1, otherwise use the private IP. I don't specifically know about Rackspace, but you might need to change the firewall settings so each server is allowed to connect to 6800. Amazon, for example, has a security group firewall, so only the servers in the security group are allowed to see 6800. Outside servers can't get to it. I'm sure Rackspace has something similar. Yes, I remember dealing with that on EC2, I'll figure out what Rackspace does. I did find the two IPs, internal and external. One thing that's becoming clear is what was listening on 6800 vs 80: 6800 is for internal communication among servers. Not sure if it's used in the single-server case, but I understand. When you start Resin with start-all (or the /etc/init.d script), Resin will compare its own IP address with the configuration IP addresses in app_servers and web_servers and start itself as the local address. (Does that make sense?) Yes, I think so. After I make some dinner I'll try to go through your original email and get a couple servers running. The first server will be my load balancer, which will hit itself and the other server. MySQL can sit on the first server (although I imagine in that configuration, it might make sense for it to sit on the other server, since the first has the added burden of being the load balancer). Thanks! I'm rather excited to try running multiple machines, and see how badly my code breaks. -- Rick ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Scaling Up
On 11/8/12 7:39 PM, Rick Mann wrote: On Nov 8, 2012, at 19:28 , Scott Ferguson f...@caucho.com wrote: Yes, all the servers will have the same configuration. Okay, so, I can set one up with the config, make a snapshot of that machine, then bring up more instances from that snapshot. I'll have to figure out how to manage the IP addresses of the hosts that I bring up; might have to manually edit the config file after they're brought up. Yep. If the IP addresses are random, like Amazon, Resin has a few tricks to help, but that's a complication that's not necessary if you can rely on an IP address. Similar question for elastic servers. You do have to tweak the config for the first elastic server, but after that you can take a snapshot of the elastic server and just bring up lots of instances with the new snapshot without changing the configuration. Yes, I remember dealing with that on EC2, I'll figure out what Rackspace does. I did find the two IPs, internal and external. One thing that's becoming clear is what was listening on 6800 vs 80: 6800 is for internal communication among servers. Not sure if it's used in the single-server case, but I understand. Exactly. When you start Resin with start-all (or the /etc/init.d script), Resin will compare its own IP address with the configuration IP addresses in app_servers and web_servers and start itself as the local address. (Does that make sense?) Yes, I think so. After I make some dinner I'll try to go through your original email and get a couple servers running. The first server will be my load balancer, which will hit itself and the other server. MySQL can sit on the first server (although I imagine in that configuration, it might make sense for it to sit on the other server, since the first has the added burden of being the load balancer). Thanks! I'm rather excited to try running multiple machines, and see how badly my code breaks. Hopefully, it won't even notice :) -- Scott ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
