Re: 5.next + 4.1.x future
[ a little late here, but hey.. ] On Fri, 12 Dec 2003, David Rees wrote: | On Fri, December 12, 2003 at 2:12 pm, Adam Fisk wrote: | | I'd be happy to send my data to the group if people are interested. | Aside from memory, I was surprised to find that the effect on CPU was | negligible (not much of a benefit from no context-switching between | threads) -- CPU was virtually the same in both cases. So, the scaling | benefits on Windows basically come from not having to allocate more | memory to new threads. I'm unfortunately not as familiar with the | Tomcat code as I'd like to be, but I assume it makes intelligent use of | thread pooling, which may even the memory benefits of NIO negligible. | At the same time, though, NIO may remove some of the constraints | introduced by thread pooling, possibly allowing Tomcat to handle heavier | loads without blowing up. An optimized NIO server would if anything | out-perform a blocking server, but maybe by not that much. | | On current Linux systems, once you start getting 500+ processes/threads | active on a typical machine, you will find that context switching starts | taking up a significant amount of system time, especially if you decide to | run any system moniting tools (like ps, or top). This is better with the | upcoming 2.6 kernels, but still doesn't scale to thousands of active | threads very well. | | However, given that you need a thread anyway to server any dynamic | content, I don't see NIO helping that much for your typical web | application. I could see NIO helping scale the serving of static content | which would be useful where people are using Tomcat standalone. Maybe | someone can prove me wrong. ;-) I've also been wondering about NIO for some time, and whether a Servlet Container ever could benefit from it. What about this idea: if you have two types of threads, one set of Servlet threads, and then one (or more) threads that do all the IO, and a set of -memory buffers- inbetween, what would you get? See, the thing is about caching - and letting the Servlet distance itself from the client. The normal case is that the output stream is more or less directly attached to the client. But if you do this other thing, having a memory buffer inbetween, the servlet thread could do its stash, and exit, immediately. Hopefully it could do several such operations within -one- context switch, instead of having to IO block on every byte output. (Btw: i do realize that the Servlet spec allows for a buffer on the output stream: this is something slightly different: a forced total-buffering of the servlet's output) So, one could tag servlets with NIO-able true/false. NIO-able are servlets that typically don't produce more than 10k of data (or something like that), and that doesn't need to hang onto the client, typically a servlet outputting and flushing (thus, sending) one byte a second while it does the heavy database query. Thus, if a servlet needs this direct link to the client, or produces very large result sets, it can't be NIOed, in the first case because it -needs- this direct link between the server thread and the client, and in the second case because the memory-buffer requirement would be to heavy. If one have loads of memory, this idea could potentially speed up processing a bit by not spening an excessive amount of CPU time in context switches. One have to realize that the clients' line-speed (and slowstart on the TCP stack) often will lead to a servlet having to spend multiple context switches before being able to exit due to the blocking of the IO - this could be totally eliminated in many cases using this aproach of dividing a dynamic page generation into two totally distinct steps: the generation of the page, and the sending of that resulting page over the network. The connector architecture in earlier tomcats used to not be able to queue requests; if no worker thread was available, it immediately returned a server full response. I then advocated an approach of having a set of worker threads (a limited set), and a line of incoming requests (that then might have a higher number of slots than the number of available workers), and then a dispatcher that stands inbetween. I believe that something along the same lines have been implemented now. This suggestion here is something along the same lines - the whole idea is to reduce the real CPU time spent in context switches - a context switch isn't just the registers and memory maps, it is also flushing of the CPU's Lx caches! -- Mvh, Endre Stølsvik M[+47 93054050] F[+47 51625182] Developer @ CoreTrek AS - http://www.coretrek.com/ CoreTrek corporate portal / EIP - http://www.corelets.com/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: 5.next + 4.1.x future
Howdy, Your name seems familiar... Ah yes, the log4j TRACE discussion. Anyways, your suggestion is interesting. It seems like it'd be a lot of work, so it should only be done if the benefit is significant. What would you estimate the throughput increase over the current implementation would be (for a NIO-able servlet)? Yoav Shapira Millennium ChemInformatics -Original Message- From: Endre Stølsvik [mailto:[EMAIL PROTECTED] Sent: Thursday, December 18, 2003 4:48 AM To: Tomcat Developers List Subject: Re: 5.next + 4.1.x future [ a little late here, but hey.. ] On Fri, 12 Dec 2003, David Rees wrote: | On Fri, December 12, 2003 at 2:12 pm, Adam Fisk wrote: | | I'd be happy to send my data to the group if people are interested. | Aside from memory, I was surprised to find that the effect on CPU was | negligible (not much of a benefit from no context-switching between | threads) -- CPU was virtually the same in both cases. So, the scaling | benefits on Windows basically come from not having to allocate more | memory to new threads. I'm unfortunately not as familiar with the | Tomcat code as I'd like to be, but I assume it makes intelligent use of | thread pooling, which may even the memory benefits of NIO negligible. | At the same time, though, NIO may remove some of the constraints | introduced by thread pooling, possibly allowing Tomcat to handle heavier | loads without blowing up. An optimized NIO server would if anything | out-perform a blocking server, but maybe by not that much. | | On current Linux systems, once you start getting 500+ processes/threads | active on a typical machine, you will find that context switching starts | taking up a significant amount of system time, especially if you decide to | run any system moniting tools (like ps, or top). This is better with the | upcoming 2.6 kernels, but still doesn't scale to thousands of active | threads very well. | | However, given that you need a thread anyway to server any dynamic | content, I don't see NIO helping that much for your typical web | application. I could see NIO helping scale the serving of static content | which would be useful where people are using Tomcat standalone. Maybe | someone can prove me wrong. ;-) I've also been wondering about NIO for some time, and whether a Servlet Container ever could benefit from it. What about this idea: if you have two types of threads, one set of Servlet threads, and then one (or more) threads that do all the IO, and a set of -memory buffers- inbetween, what would you get? See, the thing is about caching - and letting the Servlet distance itself from the client. The normal case is that the output stream is more or less directly attached to the client. But if you do this other thing, having a memory buffer inbetween, the servlet thread could do its stash, and exit, immediately. Hopefully it could do several such operations within -one- context switch, instead of having to IO block on every byte output. (Btw: i do realize that the Servlet spec allows for a buffer on the output stream: this is something slightly different: a forced total-buffering of the servlet's output) So, one could tag servlets with NIO-able true/false. NIO-able are servlets that typically don't produce more than 10k of data (or something like that), and that doesn't need to hang onto the client, typically a servlet outputting and flushing (thus, sending) one byte a second while it does the heavy database query. Thus, if a servlet needs this direct link to the client, or produces very large result sets, it can't be NIOed, in the first case because it -needs- this direct link between the server thread and the client, and in the second case because the memory-buffer requirement would be to heavy. If one have loads of memory, this idea could potentially speed up processing a bit by not spening an excessive amount of CPU time in context switches. One have to realize that the clients' line-speed (and slowstart on the TCP stack) often will lead to a servlet having to spend multiple context switches before being able to exit due to the blocking of the IO - this could be totally eliminated in many cases using this aproach of dividing a dynamic page generation into two totally distinct steps: the generation of the page, and the sending of that resulting page over the network. The connector architecture in earlier tomcats used to not be able to queue requests; if no worker thread was available, it immediately returned a server full response. I then advocated an approach of having a set of worker threads (a limited set), and a line of incoming requests (that then might have a higher number of slots than the number of available workers), and then a dispatcher that stands inbetween. I believe that something along the same lines have been implemented now. This suggestion here is something along the same lines - the whole idea is to reduce the real CPU time spent in context switches - a context
Re: 5.next + 4.1.x future
Costin Manolache wrote: Jan-Henrik Haukeland wrote: Remy Maucherat [EMAIL PROTECTED] writes: My opinion is that NIO is going to be really useless. Eh, hello!? Oh, okay if it's not important that Tomcat scale and perform well it may be useless. But, really, before NIO it was hopeless to try and write a scalable and fast tcp server application in Java. Tomcat's current connection handling with blocing all over the place and thundering herd problem doesn't scale or work very well under heavy load. Only thing you can save is the static servlet - but for that you are better off using a real server or a cache. But as Remy mentioned: Static content is handled with a regular (well, almost) servlet, and all those requests have to go through the filter pipeline (ie, user components). This is the exact same problem when having Apache serve static files. So unless you break the spec, you still will need to spawn a thread to process static content through the filter pipeline. If that happens to be OK for your application, then you are right, better to use a real server or cache in front of your application (like Squid www.squid-cache.org). -Dave - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
Remy Maucherat wrote: I'm interested in the following enhancements to 5.0.x in the future (= in january or later): - refactoring of the save to XML feature (that's been requested; I don't know if I'll use that to be able to use the admin webapp under JBoss, though); likely the default impl will remain the current one, but we'll be able to take out all that code from StandardServer :) IMO the current one is pretty bad. There is a beginning in commons-modeler, and IMO it is the right solution for the problem ( as it tracks changes to the config - instead of trying to generate a new file, and preserves comments and structure ). I don't know when things will slow down a bit at work - but in january I'll try my best to free some time. Costin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
Remy Maucherat wrote: Jeanfrancois Arcand wrote: +1 I would also like to explore, as a separate module, a connector that uses NIO to see what kind of performance we can have. I have no intention of replacing the current connector, and probably I will waste my time trying to have the same performance as the current connector. But I would like to explore it anyway ( latter next year, after the security stuff ) My opinion is that NIO is going to be really useless. - HTTP processing time is insignificant (BTW, most of the current algorithm can be used in non blocking mode, I think) - You have to go in blocking mode for the whole Catalina pipeline (ok: fast) and for running the user application - It will make object reuse more complex (you'll have to have a pool holding all the HTTP processors, essentially) NIO may be good in mod_jk mode - not the select but the mmap. I agree that select can't help too much given the servlet IO model, it's the mmap ( and the char-byte conversion ) that are really valuable in our case. BTW - another intersting optimization is for JSP static content, which can be also mmapped to avoid sending it over ( and maybe even get it sent directly from the disk ). Costin So the benefit is for holding HTTP/1.1 persistent connections, given that: - It will make the connector code more complex ( :-( ) - Modern OSes (Linux 2.6, Solaris with Sun's VMs, any Linux with JRockit and its thread mapping similar to Solaris) will probably not really care about saving a few threads But, hey, if you have time, it's an interesting experiment (and I'll try to help out a bit if I have time) :) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
Jan-Henrik Haukeland wrote:
Remy Maucherat [EMAIL PROTECTED] writes:
My opinion is that NIO is going to be really useless.
Eh, hello!? Oh, okay if it's not important that Tomcat scale and
perform well it may be useless. But, really, before NIO it was
hopeless to try and write a scalable and fast tcp server application
in Java. Tomcat's current connection handling with blocing all over
the place and thundering herd problem doesn't scale or work very
well under heavy load.
Don't forget that servlets ( which is the main job of tomcat ) use
blocking input/output streams.
NIO select ( which is what most people see first in NIO ) is not going
to help in this. Select is extremely powerfull - but it requires a
certain kind of event-based architecture which doesn't match the
blocking model of servlets ( which has the big benefit of making things
simple for users ).
Think about it - in any case you do need one thread for each active
request, for the servlet. Most of the time is spent in the service()
method - you can do select() as much as you want in http/jk processing,
but you must have the thread for service() ( which can't be avoided
since servlets are allowed to block the thread ).
Only thing you can save is the static servlet - but for that you are
better off using a real server or a cache.
It's sad people see select() first - and don't pay attention to
char-byte convertors in NIO ( a huge ammount of optimization - tomcat
already has something that emulates the same model, so we won't benefit
much ), and the mmapping feature - which is huge for IPC and java-native
communication.
Costin
- It will make object reuse more complex (you'll have to have a pool
holding all the HTTP processors, essentially)
You will have a pool holding incomming connection, and if there is an
i/o event on the connection such as data present you will hand the
connection over to a small pool of HTTP processors (5-10). Here's a
quasi example for the main server loop. (All sockets must be
non-blocking, including the serversocket, i.e. it must *not* block on
accept(). Every i/o operation must of course also be non-blocking).
This loop allows the server to handle many thousand concurrent
connections without using many threads.
for(;;) {
/* Poll the connection pool, including the server socket and
* return the number of sockets ready for processing */
numReady= Poll.poll();
if(Poll.hasConnection()) {
/* We have new connections on the server socket; accept
* every incomming connection and put them into the
* connection pool */
doAccept();
/* Continue checking for more incomming connection;
* new connections are more important than old */
continue;
}
if(numReady == 0) {
/* Close and remove connections that has timed out or
* are done */
Poll.checkTimeout();
/* Continue checking for more incomming connection */
continue;
}
/* Get and process only those connections that is
* ready. I.e. has data avilable. (More optimizing is to only
* hand over incomming connection that has read and buffered a
* complete HTTP request */
while((connection= Poll.next())) {
/* Push the connection into our HttpProcessor thread
* worker/queue for processing.
- Keep-alive connections are not closed but keept in
the connection pool and will be re-processed.
- Connections that are finished are closed by the
HttpProcessor and will be removed from the pool
in the test above */
if(! processorPush(connection)) {
/* If no more processor threads are available
* we process the request ourself. Alternatively
* increase the pool of processor threads */
processConnection(connection, NO_KEEPALIVE);
}
}
}
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
Costin Manolache wrote: Jan-Henrik Haukeland wrote: Don't forget that servlets ( which is the main job of tomcat ) use blocking input/output streams. NIO select ( which is what most people see first in NIO ) is not going to help in this. Select is extremely powerfull - but it requires a certain kind of event-based architecture which doesn't match the blocking model of servlets ( which has the big benefit of making things simple for users ). Think about it - in any case you do need one thread for each active request, for the servlet. Most of the time is spent in the service() method - you can do select() as much as you want in http/jk processing, but you must have the thread for service() ( which can't be avoided since servlets are allowed to block the thread ). Only thing you can save is the static servlet - but for that you are better off using a real server or a cache. It's sad people see select() first - and don't pay attention to char-byte convertors in NIO ( a huge ammount of optimization - tomcat already has something that emulates the same model, so we won't benefit much ), and the mmapping feature - which is huge for IPC and java-native communication. Well, we're using the C2B from NIO already. The old stuff is just wrappers around them, AFAIK. Of course, it would be more efficient to bypass the wrappers completely :) For mem maps, I don't know. The direct buffers are rather annoying to manipulate, it seems. It's good that we have a very efficient HTTP stack already, and we don't have to rush this stuff :-D Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
David Rees wrote: On Fri, December 12, 2003 at 2:12 pm, Adam Fisk wrote: I'd be happy to send my data to the group if people are interested. Aside from memory, I was surprised to find that the effect on CPU was negligible (not much of a benefit from no context-switching between threads) -- CPU was virtually the same in both cases. So, the scaling benefits on Windows basically come from not having to allocate more memory to new threads. I'm unfortunately not as familiar with the Tomcat code as I'd like to be, but I assume it makes intelligent use of thread pooling, which may even the memory benefits of NIO negligible. At the same time, though, NIO may remove some of the constraints introduced by thread pooling, possibly allowing Tomcat to handle heavier loads without blowing up. An optimized NIO server would if anything out-perform a blocking server, but maybe by not that much. On current Linux systems, once you start getting 500+ processes/threads active on a typical machine, you will find that context switching starts taking up a significant amount of system time, especially if you decide to run any system moniting tools (like ps, or top). This is better with the upcoming 2.6 kernels, but still doesn't scale to thousands of active threads very well. For purely switching, the benches I saw saw a linear scaling for the amount on processes in Linnux 2.6. I didn't try it, though. There would be a significant amount of threads in wait mode. You can't have thousands of threads actually processing stuff anyway, you'll have problems with your database first, or your CPU in the case of XML. I really wonder sometimes why some people think that NIO would make J2EE scale automagically. It could be a decent optimization for resources use (memory and some threads), but that's it, and it remains to be seen if it wouldn't cause a throughtput drop (in that case, it would be useless, given the amount of memory efficient VMs use to provide a similar throughtput increase). However, given that you need a thread anyway to server any dynamic content, I don't see NIO helping that much for your typical web application. I could see NIO helping scale the serving of static content which would be useful where people are using Tomcat standalone. Maybe someone can prove me wrong. ;-) Static content is handled with a regular (well, almost) servlet, and all those requests have to go through the filter pipeline (ie, user components). This is the exact same problem when having Apache serve static files. Nice try ;-) Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
Remy Maucherat wrote: Hi, I see no reason to create a new branch, which is not what happened with previous releases, where branches were created sometimes even before the stable release (4.1.x got forked for 4.0 final, and 5.0.x was forked for 4.1.7). Reasons for no branching: - Tomcat is rather modular - Behavior in nearly all cases seems well established, so no major breakage is to be expected - Increased maintenance requirements due to diverting resource (we did have a problem with that, looking at all the BZ items) - Branching hasn't really been synonymous with stability or quality in the past, given patch porting decisions have been questionable sometimes (that's likely my fault, as the RM) We now have a stable 5.0.16, which so far seems like a quality release. Since 5.0.x is only an evolution of 4.1.x, I don't see much needs for more than one additional 4.1.x releases (assuming it's a good one without regressions :D ). If people want more, a new RM will need to pop up :) (BTW, I can pass the hot potato right now for 4.x if someone is interested :D ) I'm interested in the following enhancements to 5.0.x in the future (= in january or later): - refactoring of the save to XML feature (that's been requested; I don't know if I'll use that to be able to use the admin webapp under JBoss, though); likely the default impl will remain the current one, but we'll be able to take out all that code from StandardServer :) - full support for a JMX-ized server.xml, which is used in the embedded distribution (I think a few components can't be created using that; I could be wrong though, Costin did a great job) - I'll add the minimal server.xml to embedded and some startup scripts and the native wrappers, and I'll optimize it some more, so that it can also be used as a minimal Tomcat installation (without any CL hierarchy, which sometimes confuses beginners) - tweaks and fixes +1 I would also like to explore, as a separate module, a connector that uses NIO to see what kind of performance we can have. I have no intention of replacing the current connector, and probably I will waste my time trying to have the same performance as the current connector. But I would like to explore it anyway ( latter next year, after the security stuff ) -- Jeanfrancois Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
Jeanfrancois Arcand wrote: +1 I would also like to explore, as a separate module, a connector that uses NIO to see what kind of performance we can have. I have no intention of replacing the current connector, and probably I will waste my time trying to have the same performance as the current connector. But I would like to explore it anyway ( latter next year, after the security stuff ) My opinion is that NIO is going to be really useless. - HTTP processing time is insignificant (BTW, most of the current algorithm can be used in non blocking mode, I think) - You have to go in blocking mode for the whole Catalina pipeline (ok: fast) and for running the user application - It will make object reuse more complex (you'll have to have a pool holding all the HTTP processors, essentially) So the benefit is for holding HTTP/1.1 persistent connections, given that: - It will make the connector code more complex ( :-( ) - Modern OSes (Linux 2.6, Solaris with Sun's VMs, any Linux with JRockit and its thread mapping similar to Solaris) will probably not really care about saving a few threads But, hey, if you have time, it's an interesting experiment (and I'll try to help out a bit if I have time) :) Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
Remy Maucherat [EMAIL PROTECTED] writes:
My opinion is that NIO is going to be really useless.
Eh, hello!? Oh, okay if it's not important that Tomcat scale and
perform well it may be useless. But, really, before NIO it was
hopeless to try and write a scalable and fast tcp server application
in Java. Tomcat's current connection handling with blocing all over
the place and thundering herd problem doesn't scale or work very
well under heavy load.
- It will make object reuse more complex (you'll have to have a pool
holding all the HTTP processors, essentially)
You will have a pool holding incomming connection, and if there is an
i/o event on the connection such as data present you will hand the
connection over to a small pool of HTTP processors (5-10). Here's a
quasi example for the main server loop. (All sockets must be
non-blocking, including the serversocket, i.e. it must *not* block on
accept(). Every i/o operation must of course also be non-blocking).
This loop allows the server to handle many thousand concurrent
connections without using many threads.
for(;;) {
/* Poll the connection pool, including the server socket and
* return the number of sockets ready for processing */
numReady= Poll.poll();
if(Poll.hasConnection()) {
/* We have new connections on the server socket; accept
* every incomming connection and put them into the
* connection pool */
doAccept();
/* Continue checking for more incomming connection;
* new connections are more important than old */
continue;
}
if(numReady == 0) {
/* Close and remove connections that has timed out or
* are done */
Poll.checkTimeout();
/* Continue checking for more incomming connection */
continue;
}
/* Get and process only those connections that is
* ready. I.e. has data avilable. (More optimizing is to only
* hand over incomming connection that has read and buffered a
* complete HTTP request */
while((connection= Poll.next())) {
/* Push the connection into our HttpProcessor thread
* worker/queue for processing.
- Keep-alive connections are not closed but keept in
the connection pool and will be re-processed.
- Connections that are finished are closed by the
HttpProcessor and will be removed from the pool
in the test above */
if(! processorPush(connection)) {
/* If no more processor threads are available
* we process the request ourself. Alternatively
* increase the pool of processor threads */
processConnection(connection, NO_KEEPALIVE);
}
}
}
--
Jan-Henrik Haukeland
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
Jan-Henrik Haukeland wrote: Remy Maucherat [EMAIL PROTECTED] writes: My opinion is that NIO is going to be really useless. Eh, hello!? Oh, okay if it's not important that Tomcat scale and perform well it may be useless. But, really, before NIO it was hopeless to try and write a scalable and fast tcp server application in Java. Tomcat's current connection handling with blocing all over the place and thundering herd problem doesn't scale or work very well under heavy load. You apparently have a very strong opinion on this, and that's fine. You also obviously don't know what you are talking about. The purpose of Tomcat is to make the web tier of an application server (Tomcat is actually a mini application server), not some kind of non blocking I/O toolkit to be used to build fixed function servers. Non blocking I/O has great applications, and is a very useful technology, but it does not apply to the application server world. I think you should find a servlet container which has NIO, compare with Tomcat 5.0.16, and come back to report your findings about how much scalability or speed NIO brings (note: doing the non blocking socket handling in a native layer doesn't really count, since it's not a fair comparison with Java's NIO; you might as well use Apache). Bring facts, not useless rants. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
Remy Maucherat wrote: Jan-Henrik Haukeland wrote: Remy Maucherat [EMAIL PROTECTED] writes: My opinion is that NIO is going to be really useless. Eh, hello!? Oh, okay if it's not important that Tomcat scale and perform well it may be useless. But, really, before NIO it was hopeless to try and write a scalable and fast tcp server application in Java. Tomcat's current connection handling with blocing all over the place and thundering herd problem doesn't scale or work very well under heavy load. You apparently have a very strong opinion on this, and that's fine. You also obviously don't know what you are talking about. The purpose of Tomcat is to make the web tier of an application server (Tomcat is actually a mini application server), not some kind of non blocking I/O toolkit to be used to build fixed function servers. Non blocking I/O has great applications, and is a very useful technology, but it does not apply to the application server world. I think you should find a servlet container which has NIO, compare with Tomcat 5.0.16, and come back to report your findings about how much scalability or speed NIO brings (note: doing the non blocking socket handling in a native layer doesn't really count, since it's not a fair comparison with Java's NIO; you might as well use Apache). Bring facts, not useless rants. I've lost the reference but did anyone else see the non-blocking NIO plug-in that was supposed to fit right into Java things like Tomcat and solve all their performance/scalability issues? I saw this touted somewhere recently. It sounded *way* too good to be true, but was interesting. I'm thinking it was called something like JEngine or something equally vague... What little I could make out from the info at the time seemed to suggest that it might be nice (assuming it worked) if you wanted to allow Apache to keep loads of connections open to a non-blocking IO layer in Tomcat without burdening it as much as this would normally imply. Of course the next layer of Tomcat needs to wait on the thread pool anyway, so I'm not sure what the real benefit over a reasonably sized mod_jk[2] connection pool really is... -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
Just to chime in on the NIO issue, I agree that it's not immediately obvious what the performance benefits are. Perhaps more importantly, though, the code changes to switch Tomcat (or any other good-size app) to NIO are tremendous -- basically a rewrite of the hard parts. That said, I've done simple tests of NIO vs. blocking IO on Windows using simple blocking and non-blocking servers with variable numbers of client connections (from 100 to 10,000). At least on Windows, the performance benefits come down to memory allocated to threads. My blocking server used one send thread and one receive thread per connection (no thread pooling). Given the memory allocations per thread, though, the blocking server with 1600 connections used 137MB whereas the NIO server used 11MB, almost exactly the same as the memory use with 50 connections. I'd be happy to send my data to the group if people are interested. Aside from memory, I was surprised to find that the effect on CPU was negligible (not much of a benefit from no context-switching between threads) -- CPU was virtually the same in both cases. So, the scaling benefits on Windows basically come from not having to allocate more memory to new threads. I'm unfortunately not as familiar with the Tomcat code as I'd like to be, but I assume it makes intelligent use of thread pooling, which may even the memory benefits of NIO negligible. At the same time, though, NIO may remove some of the constraints introduced by thread pooling, possibly allowing Tomcat to handle heavier loads without blowing up. An optimized NIO server would if anything out-perform a blocking server, but maybe by not that much. -Adam Remy Maucherat wrote: Jan-Henrik Haukeland wrote: Remy Maucherat [EMAIL PROTECTED] writes: My opinion is that NIO is going to be really useless. Eh, hello!? Oh, okay if it's not important that Tomcat scale and perform well it may be useless. But, really, before NIO it was hopeless to try and write a scalable and fast tcp server application in Java. Tomcat's current connection handling with blocing all over the place and thundering herd problem doesn't scale or work very well under heavy load. You apparently have a very strong opinion on this, and that's fine. You also obviously don't know what you are talking about. The purpose of Tomcat is to make the web tier of an application server (Tomcat is actually a mini application server), not some kind of non blocking I/O toolkit to be used to build fixed function servers. Non blocking I/O has great applications, and is a very useful technology, but it does not apply to the application server world. I think you should find a servlet container which has NIO, compare with Tomcat 5.0.16, and come back to report your findings about how much scalability or speed NIO brings (note: doing the non blocking socket handling in a native layer doesn't really count, since it's not a fair comparison with Java's NIO; you might as well use Apache). Bring facts, not useless rants. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] . - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
Adam Fisk wrote: Just to chime in on the NIO issue, I agree that it's not immediately obvious what the performance benefits are. Perhaps more importantly, though, the code changes to switch Tomcat (or any other good-size app) to NIO are tremendous -- basically a rewrite of the hard parts. That said, I've done simple tests of NIO vs. blocking IO on Windows using simple blocking and non-blocking servers with variable numbers of client connections (from 100 to 10,000). At least on Windows, the performance benefits come down to memory allocated to threads. My blocking server used one send thread and one receive thread per connection (no thread pooling). Given the memory allocations per thread, though, the blocking server with 1600 connections used 137MB whereas the NIO server used 11MB, almost exactly the same as the memory use with 50 connections. I'd be happy to send my data to the group if people are interested. Yes, I will be very interested to see those numbers. Thanks, -- Jeanfrancois Aside from memory, I was surprised to find that the effect on CPU was negligible (not much of a benefit from no context-switching between threads) -- CPU was virtually the same in both cases. So, the scaling benefits on Windows basically come from not having to allocate more memory to new threads. I'm unfortunately not as familiar with the Tomcat code as I'd like to be, but I assume it makes intelligent use of thread pooling, which may even the memory benefits of NIO negligible. At the same time, though, NIO may remove some of the constraints introduced by thread pooling, possibly allowing Tomcat to handle heavier loads without blowing up. An optimized NIO server would if anything out-perform a blocking server, but maybe by not that much. -Adam Remy Maucherat wrote: Jan-Henrik Haukeland wrote: Remy Maucherat [EMAIL PROTECTED] writes: My opinion is that NIO is going to be really useless. Eh, hello!? Oh, okay if it's not important that Tomcat scale and perform well it may be useless. But, really, before NIO it was hopeless to try and write a scalable and fast tcp server application in Java. Tomcat's current connection handling with blocing all over the place and thundering herd problem doesn't scale or work very well under heavy load. You apparently have a very strong opinion on this, and that's fine. You also obviously don't know what you are talking about. The purpose of Tomcat is to make the web tier of an application server (Tomcat is actually a mini application server), not some kind of non blocking I/O toolkit to be used to build fixed function servers. Non blocking I/O has great applications, and is a very useful technology, but it does not apply to the application server world. I think you should find a servlet container which has NIO, compare with Tomcat 5.0.16, and come back to report your findings about how much scalability or speed NIO brings (note: doing the non blocking socket handling in a native layer doesn't really count, since it's not a fair comparison with Java's NIO; you might as well use Apache). Bring facts, not useless rants. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] . - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
Here's my excel spreadsheet. All of these numbers are taken from the Windows Task Manager. This is somewhat misleading in that the memory numbers don't properly reflect garbage collection, but it's informative nevertheless. The Sends (ms) column reflects the timeout between client sends. I originally intended to run these tests for many timeouts, but this became prohibitively annoying over the WAN. These tests were done with all clients running on a single machine connecting to the servers over the WAN. Thanks. -Adam Jeanfrancois Arcand wrote: Adam Fisk wrote: Just to chime in on the NIO issue, I agree that it's not immediately obvious what the performance benefits are. Perhaps more importantly, though, the code changes to switch Tomcat (or any other good-size app) to NIO are tremendous -- basically a rewrite of the hard parts. That said, I've done simple tests of NIO vs. blocking IO on Windows using simple blocking and non-blocking servers with variable numbers of client connections (from 100 to 10,000). At least on Windows, the performance benefits come down to memory allocated to threads. My blocking server used one send thread and one receive thread per connection (no thread pooling). Given the memory allocations per thread, though, the blocking server with 1600 connections used 137MB whereas the NIO server used 11MB, almost exactly the same as the memory use with 50 connections. I'd be happy to send my data to the group if people are interested. Yes, I will be very interested to see those numbers. Thanks, -- Jeanfrancois Aside from memory, I was surprised to find that the effect on CPU was negligible (not much of a benefit from no context-switching between threads) -- CPU was virtually the same in both cases. So, the scaling benefits on Windows basically come from not having to allocate more memory to new threads. I'm unfortunately not as familiar with the Tomcat code as I'd like to be, but I assume it makes intelligent use of thread pooling, which may even the memory benefits of NIO negligible. At the same time, though, NIO may remove some of the constraints introduced by thread pooling, possibly allowing Tomcat to handle heavier loads without blowing up. An optimized NIO server would if anything out-perform a blocking server, but maybe by not that much. -Adam Remy Maucherat wrote: Jan-Henrik Haukeland wrote: Remy Maucherat [EMAIL PROTECTED] writes: My opinion is that NIO is going to be really useless. Eh, hello!? Oh, okay if it's not important that Tomcat scale and perform well it may be useless. But, really, before NIO it was hopeless to try and write a scalable and fast tcp server application in Java. Tomcat's current connection handling with blocing all over the place and thundering herd problem doesn't scale or work very well under heavy load. You apparently have a very strong opinion on this, and that's fine. You also obviously don't know what you are talking about. The purpose of Tomcat is to make the web tier of an application server (Tomcat is actually a mini application server), not some kind of non blocking I/O toolkit to be used to build fixed function servers. Non blocking I/O has great applications, and is a very useful technology, but it does not apply to the application server world. I think you should find a servlet container which has NIO, compare with Tomcat 5.0.16, and come back to report your findings about how much scalability or speed NIO brings (note: doing the non blocking socket handling in a native layer doesn't really count, since it's not a fair comparison with Java's NIO; you might as well use Apache). Bring facts, not useless rants. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] . - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] . blocking_vs_non-blocking.xls Description: MS-Excel spreadsheet - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
I should also mention that I ran these tests on an Athlon 2200 with 512MB RAM. -Adam Adam Fisk wrote: Here's my excel spreadsheet. All of these numbers are taken from the Windows Task Manager. This is somewhat misleading in that the memory numbers don't properly reflect garbage collection, but it's informative nevertheless. The Sends (ms) column reflects the timeout between client sends. I originally intended to run these tests for many timeouts, but this became prohibitively annoying over the WAN. These tests were done with all clients running on a single machine connecting to the servers over the WAN. Thanks. -Adam Jeanfrancois Arcand wrote: Adam Fisk wrote: Just to chime in on the NIO issue, I agree that it's not immediately obvious what the performance benefits are. Perhaps more importantly, though, the code changes to switch Tomcat (or any other good-size app) to NIO are tremendous -- basically a rewrite of the hard parts. That said, I've done simple tests of NIO vs. blocking IO on Windows using simple blocking and non-blocking servers with variable numbers of client connections (from 100 to 10,000). At least on Windows, the performance benefits come down to memory allocated to threads. My blocking server used one send thread and one receive thread per connection (no thread pooling). Given the memory allocations per thread, though, the blocking server with 1600 connections used 137MB whereas the NIO server used 11MB, almost exactly the same as the memory use with 50 connections. I'd be happy to send my data to the group if people are interested. Yes, I will be very interested to see those numbers. Thanks, -- Jeanfrancois Aside from memory, I was surprised to find that the effect on CPU was negligible (not much of a benefit from no context-switching between threads) -- CPU was virtually the same in both cases. So, the scaling benefits on Windows basically come from not having to allocate more memory to new threads. I'm unfortunately not as familiar with the Tomcat code as I'd like to be, but I assume it makes intelligent use of thread pooling, which may even the memory benefits of NIO negligible. At the same time, though, NIO may remove some of the constraints introduced by thread pooling, possibly allowing Tomcat to handle heavier loads without blowing up. An optimized NIO server would if anything out-perform a blocking server, but maybe by not that much. -Adam Remy Maucherat wrote: Jan-Henrik Haukeland wrote: Remy Maucherat [EMAIL PROTECTED] writes: My opinion is that NIO is going to be really useless. Eh, hello!? Oh, okay if it's not important that Tomcat scale and perform well it may be useless. But, really, before NIO it was hopeless to try and write a scalable and fast tcp server application in Java. Tomcat's current connection handling with blocing all over the place and thundering herd problem doesn't scale or work very well under heavy load. You apparently have a very strong opinion on this, and that's fine. You also obviously don't know what you are talking about. The purpose of Tomcat is to make the web tier of an application server (Tomcat is actually a mini application server), not some kind of non blocking I/O toolkit to be used to build fixed function servers. Non blocking I/O has great applications, and is a very useful technology, but it does not apply to the application server world. I think you should find a servlet container which has NIO, compare with Tomcat 5.0.16, and come back to report your findings about how much scalability or speed NIO brings (note: doing the non blocking socket handling in a native layer doesn't really count, since it's not a fair comparison with Java's NIO; you might as well use Apache). Bring facts, not useless rants. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] . - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] . - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 5.next + 4.1.x future
On Fri, December 12, 2003 at 2:12 pm, Adam Fisk wrote: I'd be happy to send my data to the group if people are interested. Aside from memory, I was surprised to find that the effect on CPU was negligible (not much of a benefit from no context-switching between threads) -- CPU was virtually the same in both cases. So, the scaling benefits on Windows basically come from not having to allocate more memory to new threads. I'm unfortunately not as familiar with the Tomcat code as I'd like to be, but I assume it makes intelligent use of thread pooling, which may even the memory benefits of NIO negligible. At the same time, though, NIO may remove some of the constraints introduced by thread pooling, possibly allowing Tomcat to handle heavier loads without blowing up. An optimized NIO server would if anything out-perform a blocking server, but maybe by not that much. On current Linux systems, once you start getting 500+ processes/threads active on a typical machine, you will find that context switching starts taking up a significant amount of system time, especially if you decide to run any system moniting tools (like ps, or top). This is better with the upcoming 2.6 kernels, but still doesn't scale to thousands of active threads very well. However, given that you need a thread anyway to server any dynamic content, I don't see NIO helping that much for your typical web application. I could see NIO helping scale the serving of static content which would be useful where people are using Tomcat standalone. Maybe someone can prove me wrong. ;-) -Dave - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: 5.next + 4.1.x future
The tomcat clustering uses NIO for the session replication, here is another fact, NIO sucks pretty bad in almost all the VMs. In some of them, it just doesn't work as advertised at all. I agree with Remy, it will not change the scalability that tomcat currently supports, because of the nature of the HTTP protocol. What you should be doing for a heavy load is to turn off keep alive for one thing, so that you don't have threads waiting for nothing. Filip -Original Message- From: Remy Maucherat [mailto:[EMAIL PROTECTED] Sent: Friday, December 12, 2003 1:38 PM To: Tomcat Developers List Subject: Re: 5.next + 4.1.x future Jan-Henrik Haukeland wrote: Remy Maucherat [EMAIL PROTECTED] writes: My opinion is that NIO is going to be really useless. Eh, hello!? Oh, okay if it's not important that Tomcat scale and perform well it may be useless. But, really, before NIO it was hopeless to try and write a scalable and fast tcp server application in Java. Tomcat's current connection handling with blocing all over the place and thundering herd problem doesn't scale or work very well under heavy load. You apparently have a very strong opinion on this, and that's fine. You also obviously don't know what you are talking about. The purpose of Tomcat is to make the web tier of an application server (Tomcat is actually a mini application server), not some kind of non blocking I/O toolkit to be used to build fixed function servers. Non blocking I/O has great applications, and is a very useful technology, but it does not apply to the application server world. I think you should find a servlet container which has NIO, compare with Tomcat 5.0.16, and come back to report your findings about how much scalability or speed NIO brings (note: doing the non blocking socket handling in a native layer doesn't really count, since it's not a fair comparison with Java's NIO; you might as well use Apache). Bring facts, not useless rants. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
