BB, It sounds like you are talking about what used to be called "batch processing". The workload is simply kicked off from a web request. Does it (attempt) to return a request to the user? Are timeouts and issue?
As far as available RAM - the obvious solution here is to move to a 64-bit OS. We've been using AMD Opterons and SuSE Linux for high-memory Tomcat (JVM heap > 2GB) for a couple of years now. But I'm also running Windows XP _64 on my desktop and its JVM seems able to take more than 2GB also. We have a somewhat similar problem, in that we have to dynamically build an in-memory "cube" of analytical data for charting and what-if exploration of results.But our reponses typically come back in between 15 seconds and a couple of minutes. Depending on how large your session is, you may not really want to use clustering - because your sessions are replicated across all or a subset of machines. However simple load balancing with session affinity might help you scale your solution - either with multiple Tomcat's on a single machine or on separate machines. But ultimately, I think you are going down the correct path to consider spawning a separate JVM to handle this if it is a true "batch request". From what I know (please correct me if I'm wrong here) Tomcat is definitely optimized for handling larger numbers of "relatively" small requests. Its also much easier to tune Tomcat if you have a larger numbers of similarly sized workloads (requests). On mainframe transaction processing the analogy was to an hour glass - if you mixed a few pebbles in with all of the grains of sand, you ended up clogging up the works for everyone - not just yourself. This also lets you allocate CPU resources to these separate JVM as you wish - if its all in Tomcat, you have much less control. Also, if communication has to occur back to a Tomcat user, you might consider using Java Messaging Service for this - at which point you should look at Jboss or similar application server that has Tomcat and a JMS bundled together. Hope these thoughts are useful. - Richard > -----Original Message----- > From: BB Commish [mailto:[EMAIL PROTECTED] > Sent: Friday, November 18, 2005 10:50 AM > To: users@tomcat.apache.org > Subject: Managing concurrent high memory processes > > We have a Struts application running on Tomcat where a few > actions within the app can invoke processing that consumes > 300+ MB of RAM and can run for several hours. The issue is > how to make the application more scalable to accommodate > multiple concurrent 'high load' processes and also make > better use of servers with available RAM beyond the jvm > limits (we have used both Sun and JRockit jvms - typically on > Win2k3 as per customer requirements). > > Would Tomcat clustering/load balancing be suited to this > problem? Or is it more geared to managing high volumes of > requests rather than just dealing with certain high load > requests that need special attention? > > We have also considered spawning a separate jvm instance (or > retrieving from a pool more likely) to handle each high load > request with the 'normal' > requests being handled within the jvm running Tomcat. Is this > a feasible option? > > I will appreciate any comments. Thanks. > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
