Thank you very much for this exhaustive reply Christopher. 2012/12/11 Christopher Schultz <ch...@christopherschultz.net>
> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Julien, > > Warning: this is long. Like, André-or-Mark-Eggers long. > > On 12/11/12 7:30 AM, Julien Martin wrote: > > I am in reference to the following blog entry: Blog > > entry< > http://blog.springsource.org/2012/05/06/spring-mvc-3-2-preview-introducing-servlet-3-async-support > > > > > > > about Spring MVC 3.2 asynchronous support. > > > > I understand Tomcat uses a thread pool in order to serve http/web > > requests. Furthermore, the above article seems to indicate that > > Spring MVC asynchronous support relieves Tomcat's thread pool and > > allows for better concurrency in the webapp by using background > > threads for "heavy-lift" operations. > > I believe you are misinterpreting what that post has to say. It's not > that a "background" thread itself is more efficient, it's that > processing that does not need to communicate with the client can be > de-coupled from the request-processing thread-pool that exists for > that purpose. > > An example - right from the blog post - will make much more sense than > what I wrote above. Let's take the example of sending an email > message. First, some assumptions: > > 1. Sending an email takes a long time (say, 5 seconds) > 2. The client does not need confirmation that the email has been sent > > If your were to write a "classic" servlet, it would look something > like this: > > doPost() { > validateOrder(); > > queueOrder(); > > sendOrderConfirmation(); // This is the email > > response.sendRedirect("/order_complete.jsp"); > } > > Let's say that validation takes 500ms, queuing takes 800ms, and > emailing (as above) takes 5000ms. That means that the request, from > the client perspective, takes 6300ms (6.3 sec). That's a noticeable delay. > > Also, during that whole time, a single request-processing thread (from > Tomcat's thread pool) is tied-up, meaning that no other requests can > be processed by that same thread. > > If you have a thread pool of size=1 (foolish, yet instructive), it > means you can only process a single transaction every 6.3 seconds. > > Lets re-write the servlet with a background thread -- no > "asynchronous" stuff from the servlet API, but just with a simple > background thread: > > doPost() { > validateOrder(); > > queueOrder(); > > (new Thread() { > public void run() { > sendOrderConfirmation(); > } > }).start(); > > response.sendRedirect("order_complete.jsp"); > } > > So, now the email is being sent by a background thread: the response > returns to the client after 1.3 seconds which is a significant > improvement. Now, we can handle a request once every 1.3 seconds with > a request-processing thread-pool of size=1. > > Note that a better implementation of the above would be to use a > thread pool for this sort of thing instead of creating a new thread > for every request. This is what Spring provides. It's not that Spring > can do a better job of thread management, it's that Tomcat's thread > pool is special: it's the only one that can actually dispatch client > requests. Off-loading onto another thread pool for background > processing means more client requests can be handled with a smaller > (or same-sized) pool. > > Looking above, you might notice that the validateOrder() and > queueOrder() processes still take some time (1.3 seconds) to complete, > and there is no interaction with the client during that time -- the > client is just sitting there waiting for a response. Work is still > getting done on the server, of course, but there's no real reason that > the request-processing thread has to be the one doing that work: we > can delegate the entire thing to a background thread so the > request-processor thread can get back to dispatching new requests. > > This is where servlet 3.0 async comes into play. > > Let's re-write the servlet as an asynchronous one. I've never actually > written one, so I'm sure the details are going to be wrong, but the > idea is the same. This time, we'll do everything asynchronously. > > doPost() { > final AsyncContext ctx = request.startAsync(); > > (new Thread() { > public void run() { > validateOrder(); > queueOrder(); > sendOrderConfirmation(); > > ctx.getResponse().sendRedirect("/order_complete.jsp"); > > ctx.complete(); > } > }).start(); > } > > So, how what happens? When startAsync is called, an AsyncContext is > created and basically the request and response are packaged-up for > later use. The doPost method creates a new thread and starts it (or it > may start a few ms later), then returns from doPost. At this point, > the request-processor thread has only spent a few ms (let's say 5ms) > setting up the request and then it goes back into Tomcat's thread-pool > and can accept another request. Meanwhile, the "background" thread > will process the actual transaction. > > Let's assume that nothing in the run() method above interacts in any > way with the client. In the first example (no async), the client waits > the whole time for a response from the server, and the > request-processing thread does all the work. So, the client waits 6.3 > seconds and the request-processing thread is "working" for 6.3 seconds. > > In the async example, the client will probably still wait 6.3 seconds, > but the request-processing thread is back and ready for more client > requests after a tiny amount of time. Of course, the transaction is > not complete, yet. > > The background thread will run and process the transaction, including > the 5-second email process. Once the email confirmation has been sent, > the background thread "sends" a redirect and completes the async > request. I'm not sure of the exact details, here, but either the > background thread itself (via getRequest().sendRedirect()) pushes the > response back to the client, or Tomcat fetches a request-processing > thread from the pool and uses that to do the same thing. I can't see > why the background-thread wouldn't do that itself, but it's up to the > container to determine who does what. > > The point is that, when using asynchronous requests, fewer > request-processing threads can handle a whole lot of load. In the > async example, still with a thread-pool of size=1 and an async-setup > time of 5ms, that means that you can handle one client transaction > every 5ms. That's much better than every 6.3 seconds, don't you think? > > (Note that this isn't magic: if your background threads are either > limited or your system can't handle the number of transactions you are > trying to asynchronously-process, eventually you'll still have > everyone waiting 6.3 seconds no matter what). > > So, a recap of throughput (req/sec) of the above 3 implementations: > > Standard: .15873 > Background: .76923 > Async: 200.00000 > > Using asynchronous dispatching can improve our throughput a huge > number of times. > > It's worth repeating what I said earlier: if your server can't > actually handle this much load (200 emails per second, let's say), > then using asych isn't going to change anything. Honestly, this trick > only works when you have a lot of heterogeneous requests. For example, > maybe 10% of your traffic is handling orders as implemented above, > while the rest of your traffic is for much smaller sub-500ms-requests. > There's probably no reason to convert those short-running requests > into asynchronous operations. Only long-running processes with no > client interaction make any sense for this. If only 10% of your > requests are orders, that means that maybe you can process 20 orders > and 190 "small" requests per second. That's much better than, say, > waiting 6.3 seconds for a single order and then processing a single > short request, then another order and so on. > > Just remember that once all request-processing threads are tied-up > doing something, everyone else waits in line. Asynchronous request > dispatching aims to run through the line as quickly as possible. It > does *not* improve the processing time of any one transaction. > > - -chris > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) > Comment: GPGTools - http://gpgtools.org > Comment: Using GnuPG with undefined - http://www.enigmail.net/ > > iEYEAREIAAYFAlDHTdkACgkQ9CaO5/Lv0PDrNACgsaeHmBzr9RMSFuZX9ksX3g9d > bKYAniJzbqRjGBAjwxIYihvcyJYV5rIl > =l+ie > -----END PGP SIGNATURE----- > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
