Re: Tomcat in a High Traffic Environment
On Fri, 29 Oct 2004 16:39:36 -0700 (PDT), David Rees [EMAIL PROTECTED] wrote: Mladen Turk wrote: Yes, but the keepalive is used mainly for making the 'state' out of 'stateless' protocol, and it's main advantage is that you don't need to acquire a new connection all the time. Take a look at RFC2068. Even apache keeps the thread open on keepalive connections (Of course you have a KeepAliveTimeout). Without keepalive your cluster will perhaps work better in the lab, but it will fail in the real-user scenario. This is false. Your cluster will not break in a real-user scenario without Keep-Alive turned on. There are many HTTP servers out there which default to Keep-Alive turned off. Keep-Alive is not used for keeping state of a stateless protocol. Pure and simple, it is used to improve client-side performance when requesting multiple resources in a short timespan by reducing the number of TCP/IP connection starts and stops. This is mostly noticable if your website design requires the user to download a large number of small resources to view a page. It is very common to turn off Keep-Alive or significantly reduce it's timeout when attempting to scale to a large number of users without keeping a large number of usable connections idle. You can easily double the number of concurrent users handled by a server by turning off Keep-Alive. The only drawback is that if your pages have a large number of resources to retrieve per page and your users have a high-latency connection to the server. In the setup where images are served by something else, and Tomcat only handle the (much more complex) dynamic requests, then disabling keeapalive makes sense. -- x Rémy Maucherat Developer Consultant JBoss Group (Europe) SàRL x - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Tomcat in a High Traffic Environment
Dear List, I am new to the list and have a few questions about Tomcat 5.0. We are attempting to use tomcat in a High Traffic, many simultaneous Internet user environment. I have about 8000 simultaneous users, and plan on using 14 web servers. These servers connect via CORBA to a group of backend servers. The 14 web severs are load balanced with a hardware load balancer. How does Tomcat deal with connections? When a request from a user arrives, is it assigned a thread? and is this thread ONLY used for this connection, (including obtaining information from the backend) until the request is finished? How are keep-alives dealt with? When a user requests a keep-alive, is a thread held permanently for this user, until the connection is dropped? If this is the case, how is this ment to scale? A connection pool of 750 threads seems unusable... How can 1 thread per connection scale? or have I misunderstood how tomcat uses its connection pool? And should all of these threads ever have something to do at the same time, the box would just fall over with a load of 750 Would it not make more sense to use a smaller connection pool, and set up queues? Would it not then be better when the request has been processed, to put this into a second queue for requests which then go to the backend, etc, etc? So many threads can't help performance. Wouldn't the kernel be busy the whole time with context switching? and no user would ever get any data back Thanks in advance for any comments, Andrew - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
if you're using hardware load balancer like cisco localdirector, I would setup the load balancer to direct the traffic based on sessionid. this way, you don't need to use keep alive. when you say 8K simultaneous users, what does that translate to in terms of concurrent requests per second? An easy way to figure that out is to use a Http analysis tool like webtrends, or HttpAnalyze to generate statistics. I would look at the peak and average concurent requests. once you know that, it will be easier to determine if 14 servers are sufficient. peter On Fri, 29 Oct 2004 16:53:26 +0200, Andrew Miehs [EMAIL PROTECTED] wrote: Dear List, I am new to the list and have a few questions about Tomcat 5.0. We are attempting to use tomcat in a High Traffic, many simultaneous Internet user environment. I have about 8000 simultaneous users, and plan on using 14 web servers. These servers connect via CORBA to a group of backend servers. The 14 web severs are load balanced with a hardware load balancer. How does Tomcat deal with connections? When a request from a user arrives, is it assigned a thread? and is this thread ONLY used for this connection, (including obtaining information from the backend) until the request is finished? How are keep-alives dealt with? When a user requests a keep-alive, is a thread held permanently for this user, until the connection is dropped? If this is the case, how is this ment to scale? A connection pool of 750 threads seems unusable... How can 1 thread per connection scale? or have I misunderstood how tomcat uses its connection pool? And should all of these threads ever have something to do at the same time, the box would just fall over with a load of 750 Would it not make more sense to use a smaller connection pool, and set up queues? Would it not then be better when the request has been processed, to put this into a second queue for requests which then go to the backend, etc, etc? So many threads can't help performance. Wouldn't the kernel be busy the whole time with context switching? and no user would ever get any data back Thanks in advance for any comments, Andrew - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
Hi Peter, The load balancer is an F5 and we are doing can do the balancing based on JSESSIONID. At the moment we are doing persistence based on our own cookie (Long story). How did you disable keep-alives? maxKeepAliveRequests=1 ? Doesn't it make more sense to use keep-alives? And what does tomcat really do with these keep-alive connections? Does it really keep 1 thread open for each keep-alive? this seems VERY unnecessary Regards Andrew On 29.10.2004, at 17:02, Peter Lin wrote: if you're using hardware load balancer like cisco localdirector, I would setup the load balancer to direct the traffic based on sessionid. this way, you don't need to use keep alive. when you say 8K simultaneous users, what does that translate to in terms of concurrent requests per second? An easy way to figure that out is to use a Http analysis tool like webtrends, or HttpAnalyze to generate statistics. I would look at the peak and average concurent requests. once you know that, it will be easier to determine if 14 servers are sufficient. peter On Fri, 29 Oct 2004 16:53:26 +0200, Andrew Miehs [EMAIL PROTECTED] wrote: Dear List, I am new to the list and have a few questions about Tomcat 5.0. We are attempting to use tomcat in a High Traffic, many simultaneous Internet user environment. I have about 8000 simultaneous users, and plan on using 14 web servers. These servers connect via CORBA to a group of backend servers. The 14 web severs are load balanced with a hardware load balancer. How does Tomcat deal with connections? When a request from a user arrives, is it assigned a thread? and is this thread ONLY used for this connection, (including obtaining information from the backend) until the request is finished? How are keep-alives dealt with? When a user requests a keep-alive, is a thread held permanently for this user, until the connection is dropped? If this is the case, how is this ment to scale? A connection pool of 750 threads seems unusable... How can 1 thread per connection scale? or have I misunderstood how tomcat uses its connection pool? And should all of these threads ever have something to do at the same time, the box would just fall over with a load of 750 Would it not make more sense to use a smaller connection pool, and set up queues? Would it not then be better when the request has been processed, to put this into a second queue for requests which then go to the backend, etc, etc? So many threads can't help performance. Wouldn't the kernel be busy the whole time with context switching? and no user would ever get any data back Thanks in advance for any comments, Andrew - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat in a High Traffic Environment
Hi, these keep-alive connections? Does it really keep 1 thread open for each keep-alive? this seems VERY unnecessary Remember that the Servlet Spec mandates the Servlet Container service a request with one thread, independent of the HTTP details. So before you think we're clueless when it comes to performant connection handling design, please keep that in mind ;) Yoav This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
you don't need to use keepalive. generally, in a load balanced setup, keepalive is disabled because the load balancer is already making sure the user goes to the same webserver for the life time of the session. keepalive is usually set in the HTTP header by the client, so I don't think you need to do anything to tomcat's config. someone correct me if I'm wrong. remy can provide a better description of what happens in the case a browser wants to use keep alive. peter On Fri, 29 Oct 2004 17:20:18 +0200, Andrew Miehs [EMAIL PROTECTED] wrote: Hi Peter, The load balancer is an F5 and we are doing can do the balancing based on JSESSIONID. At the moment we are doing persistence based on our own cookie (Long story). How did you disable keep-alives? maxKeepAliveRequests=1 ? Doesn't it make more sense to use keep-alives? And what does tomcat really do with these keep-alive connections? Does it really keep 1 thread open for each keep-alive? this seems VERY unnecessary Regards Andrew On 29.10.2004, at 17:02, Peter Lin wrote: if you're using hardware load balancer like cisco localdirector, I would setup the load balancer to direct the traffic based on sessionid. this way, you don't need to use keep alive. when you say 8K simultaneous users, what does that translate to in terms of concurrent requests per second? An easy way to figure that out is to use a Http analysis tool like webtrends, or HttpAnalyze to generate statistics. I would look at the peak and average concurent requests. once you know that, it will be easier to determine if 14 servers are sufficient. peter On Fri, 29 Oct 2004 16:53:26 +0200, Andrew Miehs [EMAIL PROTECTED] wrote: Dear List, I am new to the list and have a few questions about Tomcat 5.0. We are attempting to use tomcat in a High Traffic, many simultaneous Internet user environment. I have about 8000 simultaneous users, and plan on using 14 web servers. These servers connect via CORBA to a group of backend servers. The 14 web severs are load balanced with a hardware load balancer. How does Tomcat deal with connections? When a request from a user arrives, is it assigned a thread? and is this thread ONLY used for this connection, (including obtaining information from the backend) until the request is finished? How are keep-alives dealt with? When a user requests a keep-alive, is a thread held permanently for this user, until the connection is dropped? If this is the case, how is this ment to scale? A connection pool of 750 threads seems unusable... How can 1 thread per connection scale? or have I misunderstood how tomcat uses its connection pool? And should all of these threads ever have something to do at the same time, the box would just fall over with a load of 750 Would it not make more sense to use a smaller connection pool, and set up queues? Would it not then be better when the request has been processed, to put this into a second queue for requests which then go to the backend, etc, etc? So many threads can't help performance. Wouldn't the kernel be busy the whole time with context switching? and no user would ever get any data back Thanks in advance for any comments, Andrew - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
Peter Lin wrote: you don't need to use keepalive. generally, in a load balanced setup, keepalive is disabled because the load balancer is already making sure the user goes to the same webserver for the life time of the session. keepalive is usually set in the HTTP header by the client, so I don't think you need to do anything to tomcat's config. someone correct me if I'm wrong. remy can provide a better description of what happens in the case a browser wants to use keep alive. Yes, but the keepalive is used mainly for making the 'state' out of 'stateless' protocol, and it's main advantage is that you don't need to acquire a new connection all the time. Take a look at RFC2068. Even apache keeps the thread open on keepalive connections (Of course you have a KeepAliveTimeout). Without keepalive your cluster will perhaps work better in the lab, but it will fail in the real-user scenario. MT. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
Hi Yoav, I have not read the Servlet Spec, so please pardon my ignorance. (Definitely do not mean to offend). What I still haven't had clearly answered is: User A sends request (with keepalive) to tomcat. Tomcat assigns request to thread (T1). Tomcat sends result back. Is thread T1 now kept reserved for user A until he/ or tomcat closes the keep-alive connection? If this is the case, and I now have 1000 users, each with 2 connections per user (ie: 2000 connections) do I need 2000 threads? What if the requests each only take a very short amount of time? The server is capable of answering a lot more requests than those from the 1000 users, but due to the number of connections and threads, I am limited to the 1000 users per machine... Or am I just missing something here... I can understand the issue with the one thread one request in the Spec - am just wondering if there isn't a better way, as I can not see how this can possible scale in a high traffic environment - or maybe I just don't understand... :-( Thanks Andrew On 29.10.2004, at 17:28, Shapira, Yoav wrote: Hi, these keep-alive connections? Does it really keep 1 thread open for each keep-alive? this seems VERY unnecessary Remember that the Servlet Spec mandates the Servlet Container service a request with one thread, independent of the HTTP details. So before you think we're clueless when it comes to performant connection handling design, please keep that in mind ;) Yoav This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
The loadbalancer forwards the packet to tomcat, and as such, tomcat sees the keep-alive request. If tomcat has keep-alive enabled, it will set up keep alive on its end. Andrew On 29.10.2004, at 17:31, Peter Lin wrote: you don't need to use keepalive. generally, in a load balanced setup, keepalive is disabled because the load balancer is already making sure the user goes to the same webserver for the life time of the session. keepalive is usually set in the HTTP header by the client, so I don't think you need to do anything to tomcat's config. someone correct me if I'm wrong. remy can provide a better description of what happens in the case a browser wants to use keep alive. peter On Fri, 29 Oct 2004 17:20:18 +0200, Andrew Miehs [EMAIL PROTECTED] wrote: Hi Peter, The load balancer is an F5 and we are doing can do the balancing based on JSESSIONID. At the moment we are doing persistence based on our own cookie (Long story). How did you disable keep-alives? maxKeepAliveRequests=1 ? Doesn't it make more sense to use keep-alives? And what does tomcat really do with these keep-alive connections? Does it really keep 1 thread open for each keep-alive? this seems VERY unnecessary Regards Andrew On 29.10.2004, at 17:02, Peter Lin wrote: if you're using hardware load balancer like cisco localdirector, I would setup the load balancer to direct the traffic based on sessionid. this way, you don't need to use keep alive. when you say 8K simultaneous users, what does that translate to in terms of concurrent requests per second? An easy way to figure that out is to use a Http analysis tool like webtrends, or HttpAnalyze to generate statistics. I would look at the peak and average concurent requests. once you know that, it will be easier to determine if 14 servers are sufficient. peter On Fri, 29 Oct 2004 16:53:26 +0200, Andrew Miehs [EMAIL PROTECTED] wrote: Dear List, I am new to the list and have a few questions about Tomcat 5.0. We are attempting to use tomcat in a High Traffic, many simultaneous Internet user environment. I have about 8000 simultaneous users, and plan on using 14 web servers. These servers connect via CORBA to a group of backend servers. The 14 web severs are load balanced with a hardware load balancer. How does Tomcat deal with connections? When a request from a user arrives, is it assigned a thread? and is this thread ONLY used for this connection, (including obtaining information from the backend) until the request is finished? How are keep-alives dealt with? When a user requests a keep-alive, is a thread held permanently for this user, until the connection is dropped? If this is the case, how is this ment to scale? A connection pool of 750 threads seems unusable... How can 1 thread per connection scale? or have I misunderstood how tomcat uses its connection pool? And should all of these threads ever have something to do at the same time, the box would just fall over with a load of 750 Would it not make more sense to use a smaller connection pool, and set up queues? Would it not then be better when the request has been processed, to put this into a second queue for requests which then go to the backend, etc, etc? So many threads can't help performance. Wouldn't the kernel be busy the whole time with context switching? and no user would ever get any data back Thanks in advance for any comments, Andrew - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
turn off keep alive - Original Message - From: Andrew Miehs [EMAIL PROTECTED] To: Tomcat Users List [EMAIL PROTECTED] Sent: Friday, October 29, 2004 10:45 AM Subject: Re: Tomcat in a High Traffic Environment Hi Yoav, I have not read the Servlet Spec, so please pardon my ignorance. (Definitely do not mean to offend). What I still haven't had clearly answered is: User A sends request (with keepalive) to tomcat. Tomcat assigns request to thread (T1). Tomcat sends result back. Is thread T1 now kept reserved for user A until he/ or tomcat closes the keep-alive connection? If this is the case, and I now have 1000 users, each with 2 connections per user (ie: 2000 connections) do I need 2000 threads? What if the requests each only take a very short amount of time? The server is capable of answering a lot more requests than those from the 1000 users, but due to the number of connections and threads, I am limited to the 1000 users per machine... Or am I just missing something here... I can understand the issue with the one thread one request in the Spec - am just wondering if there isn't a better way, as I can not see how this can possible scale in a high traffic environment - or maybe I just don't understand... :-( Thanks Andrew On 29.10.2004, at 17:28, Shapira, Yoav wrote: Hi, these keep-alive connections? Does it really keep 1 thread open for each keep-alive? this seems VERY unnecessary Remember that the Servlet Spec mandates the Servlet Container service a request with one thread, independent of the HTTP details. So before you think we're clueless when it comes to performant connection handling design, please keep that in mind ;) Yoav This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
mladen makes a good point. the sites I've worked on, we left the keepalive up to the browser and didn't explicitly disable keepalive. the sites I've worked on we simply used hardware load balancer to make sure the session goes to the right server. that is usually enough from my experience. my bias perspective is, unless the users are sending requests very rapidly, I don't see much benefit to using keepalive. If the interval between requests is rather long, like minutes, I doubt keepalive would provide any measurable benefit. I would look at the statistics from the http logs to determine whether keepalive should be disabled or not. peter On Fri, 29 Oct 2004 17:44:09 +0200, Mladen Turk [EMAIL PROTECTED] wrote: Peter Lin wrote: you don't need to use keepalive. generally, in a load balanced setup, keepalive is disabled because the load balancer is already making sure the user goes to the same webserver for the life time of the session. keepalive is usually set in the HTTP header by the client, so I don't think you need to do anything to tomcat's config. someone correct me if I'm wrong. remy can provide a better description of what happens in the case a browser wants to use keep alive. Yes, but the keepalive is used mainly for making the 'state' out of 'stateless' protocol, and it's main advantage is that you don't need to acquire a new connection all the time. Take a look at RFC2068. Even apache keeps the thread open on keepalive connections (Of course you have a KeepAliveTimeout). Without keepalive your cluster will perhaps work better in the lab, but it will fail in the real-user scenario. MT. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
Peter Lin wrote: mladen makes a good point. the sites I've worked on, we left the keepalive up to the browser and didn't explicitly disable keepalive. the sites I've worked on we simply used hardware load balancer to make sure the session goes to the right server. Correct. The keepalive is usually used for loading a single page with extra content on it, so that browser doesn't need to connect for every single icon on the web page. Of course it's also a backdor for DOS attacks, so it's up to the admin to decide the proper policy. I would look at the statistics from the http logs to determine whether keepalive should be disabled or not. Yep, me too :). MT. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
Hi Filip, Is this how you disable keep-alive on tomcat? maxKeepAliveRequests=1 or is there another switch that I am missing.. On 29.10.2004, at 17:49, Filip Hanik - Dev wrote: turn off keep alive - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
Hi Peter, I am not using keep-alives to keep session persistence, but was rather hoping for better client performance. Using keep-alives saves creating a tcp connection for each request - and thereby saving 3 tcp packets (and roundtrip times) per request. Andrew On 29.10.2004, at 17:53, Peter Lin wrote: mladen makes a good point. the sites I've worked on, we left the keepalive up to the browser and didn't explicitly disable keepalive. the sites I've worked on we simply used hardware load balancer to make sure the session goes to the right server. that is usually enough from my experience. my bias perspective is, unless the users are sending requests very rapidly, I don't see much benefit to using keepalive. If the interval between requests is rather long, like minutes, I doubt keepalive would provide any measurable benefit. I would look at the statistics from the http logs to determine whether keepalive should be disabled or not. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
On Fri, 29 Oct 2004 17:45:15 +0200, Andrew Miehs [EMAIL PROTECTED] wrote: Hi Yoav, I have not read the Servlet Spec, so please pardon my ignorance. (Definitely do not mean to offend). What I still haven't had clearly answered is: User A sends request (with keepalive) to tomcat. Tomcat assigns request to thread (T1). Tomcat sends result back. Is thread T1 now kept reserved for user A until he/ or tomcat closes the keep-alive connection? Yes. If this is the case, and I now have 1000 users, each with 2 connections per user (ie: 2000 connections) do I need 2000 threads? What if the requests each only take a very short amount of time? The server is capable of answering a lot more requests than those from the 1000 users, but due to the number of connections and threads, I am limited to the 1000 users per machine... No. If the number of used threads get higher, timeout will be reduced, and if it gets even higher, keepalive will not be used until the number of connections get lower. Or am I just missing something here... I can understand the issue with the one thread one request in the Spec - am just wondering if there isn't a better way, as I can not see how this can possible scale in a high traffic environment - or maybe I just don't understand... :-( If you have that much connections, I think a number of them will be active doing stuff anyway, so the scalability problems will not be on the number of threads. And then you'll have to experiment a little in test environments to see if disabling keepalive (yes, it's maxKeepAliveRequests=1) would be beneficial or not to the stuff you're running. Keepalive does improve performance significantly because it reduces the work of the TCP stack. BTW, Tomcat performs very well without keepalive, as there's no particular HTTP keepalive optimizations (besides benefitting from lower overhead at the TCP level). -- x Rémy Maucherat Developer Consultant JBoss Group (Europe) SàRL x - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
if you're looking for better client performance I would explore other areas first. 1. use gzip compression - this can reduce the html to 1/10th the size. your mileage will vary. 2. caching results on the web-tier 3. putting the images on a dedicated image server 4. distributing your servers across multiple ISP. many service providers don't tell you this, but often their pipe is saturated and can't really handle a large number of concurrent requests. if you host your own servers i would recommend getting more than 1 connection and use different providers Most browsers today are Http1.1 compliant, which means they are limited to 2 connections to the same server. Normally the browser will use the same connection to get the html and the other resources like images and javascript. Are you worrying about images? if that is the case, setup a dedicated image server using apache. you'll easily double the performance of your servlet pages, since tomcat won't have to serve up static content. Plus, it's easier to update the images for the entire site. Rather than copy all the files to every single webserver, you just update the image server and it's done. peter On Fri, 29 Oct 2004 18:01:29 +0200, Andrew Miehs [EMAIL PROTECTED] wrote: Hi Peter, I am not using keep-alives to keep session persistence, but was rather hoping for better client performance. Using keep-alives saves creating a tcp connection for each request - and thereby saving 3 tcp packets (and roundtrip times) per request. Andrew On 29.10.2004, at 17:53, Peter Lin wrote: - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
On 29.10.2004, at 19:08, Peter Lin wrote: if you're looking for better client performance I would explore other areas first. 1. use gzip compression - this can reduce the html to 1/10th the size. your mileage will vary. This is being looked at - loadbalancer vrs tomcat 2. caching results on the web-tier Very dynamic content 3. putting the images on a dedicated image server Already being done. 2x Servers running apache - which also have this keep-alive problem. Running 1000 threads per server is NOT my idea of a good time. I will be having a look at a couple of other alternatives to apache over the next couple of weeks. Due to operating system/ kenerl overheads time for one request (time for 10 parallel requests)/10 (time for 1000 parallel requests)/1000 Squid is a good example of how you can server MANY connections without starting thousands of threads. 4. distributing your servers across multiple ISP. many service providers don't tell you this, but often their pipe is saturated and can't really handle a large number of concurrent requests. if you host your own servers i would recommend getting more than 1 connection and use different providers This is NOT a pipe saturation issue. The issue is definitely a tomcat/ number of connections issue. I already have this problem in my local network with load tests. Most browsers today are Http1.1 compliant, which means they are limited to 2 connections to the same server. Normally the browser will use the same connection to get the html and the other resources like images and javascript. This will only happen if keep-alives are enabled. If keep-alives are disabled each GET will be a new connection. The F5 load balancers (4.5) have a 'cool-feature?!' that 'forwards' the keep-alive connection through to the backend server - With the 4.5 version of the software they are doing packet mangling. This means that even though the cllient only has 2 connections, the load balancer multiplies this connection to EACH of the backend servers with which the client is communicating, ie: static servers get 2 connections per client, statistic servers get 2 connections per client, tomcats get 2 connections per client.. Version 9 of the software has just come out, and it does proxying this should hopefully help solve this problem - I am still in the process of testing the new version. Andrew - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
Andrew Miehs wrote: 3. putting the images on a dedicated image server Already being done. 2x Servers running apache - which also have this keep-alive problem. Running 1000 threads per server is NOT my idea of a good time. I will be having a look at a couple of other alternatives to apache over the next couple of weeks. Due to operating system/ kenerl overheads time for one request (time for 10 parallel requests)/10 (time for 1000 parallel requests)/1000 Squid is a good example of how you can server MANY connections without starting thousands of threads. First of all the squid doesn't serve any content, and it can be implemented as state machine very easily. So you can not compare the proxy with the web server. There has been some attempts to make the web server as state machine, even for apache. One of the commercial one STM's is Zeus. All that looks very good in the lab environment, not so sure how it behaves in the real-life. MT. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
sounds like your goal is to maximize the number of connections to tomcat while reducing the threads. Would that be an accurate assesment? if that is the case, then I would recommend not using servlets at all. about the only way to do that would be to use a server which multiplexes n connections to a set number of threads. similar to SEDA architecture. neither .NET, J2EE or apache provide these kinds of features. I think there is a webserver based on SEDA design from a commercial company, but I don't remember the name and have never used it. out of curiousity, how big are the pages? if the html is less 1% of the entire page (html, images, javascript, etc), and you have the budget, Akamai might be a viable solution to your problem. unless your webpages use lots of frames and layers, I would guess a single request would only require one connection, since you're already using a dedicated image server. I hope that helps peter On Fri, 29 Oct 2004 19:24:07 +0200, Andrew Miehs [EMAIL PROTECTED] wrote: On 29.10.2004, at 19:08, Peter Lin wrote: if you're looking for better client performance I would explore other areas first. 1. use gzip compression - this can reduce the html to 1/10th the size. your mileage will vary. This is being looked at - loadbalancer vrs tomcat 2. caching results on the web-tier Very dynamic content 3. putting the images on a dedicated image server Already being done. 2x Servers running apache - which also have this keep-alive problem. Running 1000 threads per server is NOT my idea of a good time. I will be having a look at a couple of other alternatives to apache over the next couple of weeks. Due to operating system/ kenerl overheads time for one request (time for 10 parallel requests)/10 (time for 1000 parallel requests)/1000 Squid is a good example of how you can server MANY connections without starting thousands of threads. 4. distributing your servers across multiple ISP. many service providers don't tell you this, but often their pipe is saturated and can't really handle a large number of concurrent requests. if you host your own servers i would recommend getting more than 1 connection and use different providers This is NOT a pipe saturation issue. The issue is definitely a tomcat/ number of connections issue. I already have this problem in my local network with load tests. Most browsers today are Http1.1 compliant, which means they are limited to 2 connections to the same server. Normally the browser will use the same connection to get the html and the other resources like images and javascript. This will only happen if keep-alives are enabled. If keep-alives are disabled each GET will be a new connection. The F5 load balancers (4.5) have a 'cool-feature?!' that 'forwards' the keep-alive connection through to the backend server - With the 4.5 version of the software they are doing packet mangling. This means that even though the cllient only has 2 connections, the load balancer multiplies this connection to EACH of the backend servers with which the client is communicating, ie: static servers get 2 connections per client, statistic servers get 2 connections per client, tomcats get 2 connections per client.. Version 9 of the software has just come out, and it does proxying this should hopefully help solve this problem - I am still in the process of testing the new version. Andrew - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
Mladen Turk wrote: Yes, but the keepalive is used mainly for making the 'state' out of 'stateless' protocol, and it's main advantage is that you don't need to acquire a new connection all the time. Take a look at RFC2068. Even apache keeps the thread open on keepalive connections (Of course you have a KeepAliveTimeout). Without keepalive your cluster will perhaps work better in the lab, but it will fail in the real-user scenario. This is false. Your cluster will not break in a real-user scenario without Keep-Alive turned on. There are many HTTP servers out there which default to Keep-Alive turned off. Keep-Alive is not used for keeping state of a stateless protocol. Pure and simple, it is used to improve client-side performance when requesting multiple resources in a short timespan by reducing the number of TCP/IP connection starts and stops. This is mostly noticable if your website design requires the user to download a large number of small resources to view a page. It is very common to turn off Keep-Alive or significantly reduce it's timeout when attempting to scale to a large number of users without keeping a large number of usable connections idle. You can easily double the number of concurrent users handled by a server by turning off Keep-Alive. The only drawback is that if your pages have a large number of resources to retrieve per page and your users have a high-latency connection to the server. -Dave - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat in a High Traffic Environment
Andrew Miehs wrote: A connection pool of 750 threads seems unusable... How can 1 thread per connection scale? or have I misunderstood how tomcat uses its connection pool? And should all of these threads ever have something to do at the same time, the box would just fall over with a load of 750 Keep in mind that most of these connections will not be utilizing much CPU, they will most likely be busy sending/receiving data (or if keep-alive is on waiting doing nothing). Would it not make more sense to use a smaller connection pool, and set up queues? If you are serving resources which utilize a lot of CPU, then yes, it can be beneficial. You can use the OS's connection backlog on a listening TCP/IP socket as your queue. See the acceptCount parameter of the HTTP Connector. The default configuration has this set to 100. So if you only have 10 threads in your pool and all are busy, the OS will queue up to 100 additional connections to wait for a thread to become free. If more than 100 connections are waiting, any further connections will have their connection refused. Would it not then be better when the request has been processed, to put this into a second queue for requests which then go to the backend, etc, etc? So many threads can't help performance. Wouldn't the kernel be busy the whole time with context switching? and no user would ever get any data back Regarding kernel-switching, note my earlier comment that you will find that most connections are waiting for TCP/IP data to be either sent or received. In addition, any modern OS on decent hardware will scale to thousands of threads without any issues. -Dave - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
