RE: Multihosting in tomcat
Hi, There does not seem to be any documentation for Tomcat in a multihosting environment. The main things which are lacking are the results of setting certain parameters too high or too low. I've also mentioned MANY MANY MANY times why such documentation can be worse than useless, and actually misleading: it's specific to the application and the environment being documented. These characteristics greatly impact performance and the specific setup is just that: specific. Set it up and test/tune repeatedly until you're happy. Then if you feel like it post your findings so that other people who think they're a silver bullet might benefit. Ex. MaxProcessors (Tomcat 4.1.27) - we have an issue where one site will creep up to 107 thread and be so slow that we have to restart the service (JVM Instance.) Default value is 75 - how does it even get up to 107? We 75 is the number of processor threads. There are many others in the JVM, some related to tomcat and some that are not. You can use a profiler, a simple little utility (like the SystemThreadList.java class I've posted to the list numerous times), or a ctrl-break thread dump to see what all the threads are. have been constantly modifiying this along with accept count and monitoring trying to figure the best fit. However, I don't know if this is a good value to be modifying when you have 20+ sites. You're going through the right process, assuming you're doing it in a test environment. Also please be careful in your usage of site, instance, web application, context, and related terms. 1) Is Tomcat Standalone (4.1.27 or higher) still a viable implementation in a multi-instance scenario? Yes, we've had such setups in production for years now. 2) What are the ramifications of modifing these parameters in a multi-instance scenario? They impact performance directly. You have to test modifications out in a test environment before employing them in production. 3) Is anyone willing to join a create a discussion group on this topic or should we keep posting our thoughts/concerns here? Stay here: this is a central source for information about tomcat, it's widely archived and referenced. So far the testing of standalone tomcat proves to be ok, however, I am weary of changing production because test environment and load tests almost never provide the same results as real life :) If you have little faith in your tests, that's unfortunate. As long as they simulate why you think is a realistic load, you'll not only be fine, but you'll be covered (to higher ups) to a good extent if things go wrong. Yoav This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Multihosting in tomcat
I too have questions concerning multihosting with Tomcat. Yoav has mentioned a few times about not needing a connector to IIS or Apache. What I wonder is whether or not the Tomcat Standalone method would work in a situation where you have 20+ instances (Windows Services/JVMs) running on one installation of Tomcat. There does not seem to be any documentation for Tomcat in a multihosting environment. The main things which are lacking are the results of setting certain parameters too high or too low. Ex. MaxProcessors (Tomcat 4.1.27) - we have an issue where one site will creep up to 107 thread and be so slow that we have to restart the service (JVM Instance.) Default value is 75 - how does it even get up to 107? We have been constantly modifiying this along with accept count and monitoring trying to figure the best fit. However, I don't know if this is a good value to be modifying when you have 20+ sites. My Questions: 1) Is Tomcat Standalone (4.1.27 or higher) still a viable implementation in a multi-instance scenario? 2) What are the ramifications of modifing these parameters in a multi-instance scenario? 3) Is anyone willing to join a create a discussion group on this topic or should we keep posting our thoughts/concerns here? So far the testing of standalone tomcat proves to be ok, however, I am weary of changing production because test environment and load tests almost never provide the same results as real life :) If anyone has any experience, knowledge or thoughts - please reply. Thanks. _ Best Restaurant Giveaway Ever! Vote for your favorites for a chance to win $1 million! http://local.msn.com/special/giveaway.asp - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Multihosting in tomcat
Hi, Good luck ;) Yoav Shapira Millennium Research Informatics -Original Message- From: Michal Kwiatek [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 11, 2004 3:42 AM To: [EMAIL PROTECTED] Subject: Multihosting in tomcat Hello all! I have to set up a multihosting service based on tomcat. What I mean by multihosting is that many people are able to deploy their web application on the server. The problem is that the applications can be badly written (for instance can contain unfinite loops) and thus cause the server to go down on resources and crash. I need a solution for: (1) identifing applications consuming to much resources, (1) killing them. I know that it is possible to programatically stop a web application deployed on Tomcat. But will it work if Tomcat gets low on resources? And how will I know which application to stop in the first place? Needless to say, it has to be done automatically. Thanks very much, Michal. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Multihosting in tomcat
To add another point to my earlier post: I'm not concerned with malicious code like System.exit(1) because a custom SecurityManager will take care of that. What worries me is the abuse of resources causing the server to go low on resources and crash. Perhaps installing a separate Tomcat instance for every application is a solution? Cheers, Michal. -Original Message- From: Michal Kwiatek Sent: Tuesday, May 11, 2004 9:42 AM To: [EMAIL PROTECTED] Subject: Multihosting in tomcat Hello all! I have to set up a multihosting service based on tomcat. What I mean by multihosting is that many people are able to deploy their web application on the server. The problem is that the applications can be badly written (for instance can contain unfinite loops) and thus cause the server to go down on resources and crash. I need a solution for: (1) identifing applications consuming to much resources, (1) killing them. I know that it is possible to programatically stop a web application deployed on Tomcat. But will it work if Tomcat gets low on resources? And how will I know which application to stop in the first place? Needless to say, it has to be done automatically. Thanks very much, Michal. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Multihosting in tomcat
Michal- We're in exactly the same situation (running 4.1.x), and our solution is to isolate the most critical application(s) in their own jvm (and tomcat instance). Because everything is running as a single process, it's difficult to get very fine-grained control over an app's resource use beyond that. I imagine there are steps one could take to add some kind of instrumentation on threads, etc., but I'd also have to imagine that being an extremely heavyweight solution. If you come up with a better way of doing things, please post. Regarding your security manager comment: You shouldn't need to write a custom security manager- you probably just need to customize your policy file. You can isolate webapps by code base or signature (if they're deployed from a war file), so you're really just limited by your tolerance for keeping the policy file up-to-date. The only customization of the manager I ever needed to do was to write a kind of open manager that logged missing permissions instead of throwing exceptions: This was to do development without having to resort to the AllPermission solution that folks tend to suggest when AccessControlExceptions start popping up. Benjamin J. Armintor Systems Analyst ITS-Systems: Mainframe Group University of Texas - Austin tele: (512) 232-6562 email: [EMAIL PROTECTED] -Original Message- From: Michal Kwiatek [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 11, 2004 9:21 AM To: Tomcat Users List Subject: RE: Multihosting in tomcat To add another point to my earlier post: I'm not concerned with malicious code like System.exit(1) because a custom SecurityManager will take care of that. What worries me is the abuse of resources causing the server to go low on resources and crash. Perhaps installing a separate Tomcat instance for every application is a solution? Cheers, Michal. -Original Message- From: Michal Kwiatek Sent: Tuesday, May 11, 2004 9:42 AM To: [EMAIL PROTECTED] Subject: Multihosting in tomcat Hello all! I have to set up a multihosting service based on tomcat. What I mean by multihosting is that many people are able to deploy their web application on the server. The problem is that the applications can be badly written (for instance can contain unfinite loops) and thus cause the server to go down on resources and crash. I need a solution for: (1) identifing applications consuming to much resources, (1) killing them. I know that it is possible to programatically stop a web application deployed on Tomcat. But will it work if Tomcat gets low on resources? And how will I know which application to stop in the first place? Needless to say, it has to be done automatically. Thanks very much, Michal. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Multihosting in tomcat
Hi, To add another point to my earlier post: I'm not concerned with malicious code like System.exit(1) because a custom SecurityManager will take care of that. What worries me is the abuse of resources causing the server to go low on resources and crash. Perhaps installing a separate Tomcat instance for every application is a solution? OK, now we're getting more realistic ;) A separate tomcat instance per application means much less worrying about them killing each other. Any one app can only damage itself. This is great from a security and stability perspective, but can be a nightmare to administer (although if you have a nice JMX console, you're in much better shape as you can configure it with all your tomcat instances and control them from one place). But since you're already handling some things with a custom security manager, and are only concerned with the resources, here's one additional possible approach. Create a listener (probably the tomcat-specific Lifecycle Listener), but can be a more portable ServletContextListener tied to the admin or manager webapps for example, or to a webapp that will always be present on your server. Have this listener spawn a (daemon) thread that monitors memory usage (via Runtime.freeMemory etc.), and when it drops below a certain percentage takes action. The action can be to notify the administrator, or do tomcat specific-things (like accessing the manager webapp programmatically to stop/restart/undeploy applications). You will not know which application is causing the problem. You also won't be able to monitor CPU usage, which is the problem in cases like infinite loops. But it's something. Yoav Shapira This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Multihosting in tomcat
Hi, Oh, BTW: JDK 1.4 has a JVM option to ignore explicit System.exit calls. I forget the exact syntax but you can look it up on Sun's VM Options web page. Yoav Shapira Millennium Research Informatics -Original Message- From: Benjamin Armintor [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 11, 2004 10:32 AM To: Tomcat Users List Subject: RE: Multihosting in tomcat Michal- We're in exactly the same situation (running 4.1.x), and our solution is to isolate the most critical application(s) in their own jvm (and tomcat instance). Because everything is running as a single process, it's difficult to get very fine-grained control over an app's resource use beyond that. I imagine there are steps one could take to add some kind of instrumentation on threads, etc., but I'd also have to imagine that being an extremely heavyweight solution. If you come up with a better way of doing things, please post. Regarding your security manager comment: You shouldn't need to write a custom security manager- you probably just need to customize your policy file. You can isolate webapps by code base or signature (if they're deployed from a war file), so you're really just limited by your tolerance for keeping the policy file up-to-date. The only customization of the manager I ever needed to do was to write a kind of open manager that logged missing permissions instead of throwing exceptions: This was to do development without having to resort to the AllPermission solution that folks tend to suggest when AccessControlExceptions start popping up. Benjamin J. Armintor Systems Analyst ITS-Systems: Mainframe Group University of Texas - Austin tele: (512) 232-6562 email: [EMAIL PROTECTED] -Original Message- From: Michal Kwiatek [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 11, 2004 9:21 AM To: Tomcat Users List Subject: RE: Multihosting in tomcat To add another point to my earlier post: I'm not concerned with malicious code like System.exit(1) because a custom SecurityManager will take care of that. What worries me is the abuse of resources causing the server to go low on resources and crash. Perhaps installing a separate Tomcat instance for every application is a solution? Cheers, Michal. -Original Message- From: Michal Kwiatek Sent: Tuesday, May 11, 2004 9:42 AM To: [EMAIL PROTECTED] Subject: Multihosting in tomcat Hello all! I have to set up a multihosting service based on tomcat. What I mean by multihosting is that many people are able to deploy their web application on the server. The problem is that the applications can be badly written (for instance can contain unfinite loops) and thus cause the server to go down on resources and crash. I need a solution for: (1) identifing applications consuming to much resources, (1) killing them. I know that it is possible to programatically stop a web application deployed on Tomcat. But will it work if Tomcat gets low on resources? And how will I know which application to stop in the first place? Needless to say, it has to be done automatically. Thanks very much, Michal. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
