On 27/01/2012 23:16, bxqdev wrote: >>> 2. async servlet api is not much usefull for servers, with nio >>> connectors (tomcat-7). at least you can't image the case. >> >> I don't say that. In fact, I said exactly the opposite. > > well, then what's the case to use both of this technologies? can you > make up a theoretical case for an example?
Yes. See below. >>> 3. async servlet api is some general technology to decouple client from >>> server, when server doesn't have nio connector (server is bio-only). >> >> No. The Servlet Async API and the considerations for when one might use >> it are completely orthogonal to connector selection. > > how is that? both connector and servlet process user http request. one > by one. how are they orthogonal? The Servlet API, including the Async API only provides blocking IO. I will ignore the APR/native connector since that just complicates things further and just consider the BIO and NIO connectors below. The fundamental difference is that BIO requires one thread per connection whereas NIO requires one thread per currently processing request plus a few threads for the pollers. Each poller can monitor several thousand connections. Therefore in a situation where you have many more connections than you have current requests (think lots of HTTP keep-alive) then NIO scales significantly better than BIO. The (very small) price you pay is performance since using a poller adds overhead to request processing. [1] shows that the NIO connector uses non-blocking IO where it can but as soon as it enters the realm of the Servlet API (reading the request body and writing the request response) it has to simulate blocking IO. There is actually a complication to BIO in newer versions of Tomcat. Rather than dedicating one thread to a connection, each time Tomcat needs to read from a connection (remember this is blocking IO and there might not be anything to read if the connection is in keep-alive but the thread will then block until some data turns up or the connection times out) Tomcat obtains a thread from an executor. This raises the possibility of having more connections than you have threads. With BIO that is usually bad and results in unexpected delays in processing requests but as we'll see shortly, there are times (not many) where it might be useful. Prior to the Servlet 3.0 Async API a single container thread was responsible for all of the processing associated with a request. A few points to note at this point: - All IO is still blocking IO - The async API is only for writing data to the response. There is no "async" reading of data from the client. A typical usage of the Servlet 3.0 Async API is as follows: - request processing starts as usual on a container thread - async is started - the container thread is returned to the container's thread pool - processing continues on an application thread - the application writes some data to the response - the application passes the response back to the container for the container to finish it off on a container thread The purpose of the Servlet 3.0 Async API is to remove the one thread per currently processing request constraint. This allows applications to do things like: - implement a stock ticker with one thread writing to many responses (rather than one thread per response) - implement a chat program with one thread writing to all the clients (rather than one thread per response) If applications relay on external resources that are known to take a long time to respond, prior to the Async API developers had no choice but to simply block waiting for the external resource to respond. With the Async API, a single thread can monitor an external resource on behalf of many requests and only dispatch the request/response pair to a container thread for processing when the external resource has replied. This allows for more efficient use of container threads. Now back to the BIO complication. Lets say you have 100 container threads configured on your BIO connector and you are absolutely sure that at any one time 20% of your incoming connections would be using Servlet 3 Async and would be being handled by an application thread then you could set maxConnections to 120 and service 120 users in parallel. Your assumption regarding the 20% was wrong then you would see potentially long delays in processing incoming requests as connections with data were waiting for a container thread to become available to process it. So back to my original point. BIO vs NIO and Servlet 3 Sync vs Async are solving orthogonal problems. If you use an NIO connector that doesn't help you write a chat application more efficiently. You would still need one thread per client. For that you need the Servlet 3.0 Async API. The Async API does not help you efficiently handle many thousands of connections when the majority of them are in HTTP keep-alive. For that you need the NIO connector. Different problems, different solutions. >> I am getting rather bored with this. Please go and read up on blocking >> IO vs. non-blocking IO and then read the Async parts of the Servlet 3.0 >> specification. Then come back here and try asking some more focussed >> questions. > > i completely understand the difference of bio vs. nio. and i've already > read async servlet spec. Clearly that is not the case. If it were, we wouldn't be having this conversation. > both technologies decouple (make async) input and output. but i can't > imagine the case when the one > would need two points of decoupling on one processing chain. That is because you failed to understand what the two different technologies were doing. > 1. bio connector & sync servlet > 1.1 i guess everything is obvious Apparently not. > 2. bio connector & async servlet > 2.1 bio connector receives request and synchronously handles it to async > servlet > 2.2 async servlet starts async processing code and returns. > 2.3 bio connector waits async servlet to finish Not correct. See comments re executor above. The BIO connector will not try to read from the connection until after the response has been completed (step 2.5 below). > 2.4 async servlet fullfills response > 2.5 bio connector returns servlet to client It is the response that is returned, not the servlet. > 3. nio connector & sync servlet > 3.1 nio connector makes select and receives http request > 3.2 nio connector handles request to sync servlet > 3.3 sync servlet processes request and return response > 3.4 nio connector makes select and receives http response from sync servlet Not correct. All IO is required to be blocking at this point. See [1] and comments above on simulated blocking. > 3.5 nio connector returns servlet to client Again, it is the response rather than the servlet that is returned. > 4. nio connector & async servlet > 4.1 nio connector makes select and receives http request > 4.2 nio connector handles request to async servlet > 4.3 async servlet starts async processing code. > 4.4 async servlet fullfills response > 4.5 nio connector makes select and receives http response from async > servlet Not correct. All IO is required to be blocking at this point. See [1] and comments above on simulated blocking. > 4.6 nio connector returns servlet to client Again, it is the response rather than the servlet that is returned. > uh, that was a lot of typing :) You don't appear to understand the relationships between the blocking IO APIs provided in the Servlet API and how that data gets to the socket (via a few buffers). It looks like you are assuming that the servlet processing and the connectors exist in separate components with separate thread pools and some form of IO connection between them. That is not the case (and would be pointless and hopelessly inefficient since the Servlet API uses blocking IO). It is the threads from the connector thread pool that are responsible for the request processing. Mark [1] http://tomcat.apache.org/tomcat-7.0-doc/config/http.html#Connector_Comparison --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
