Ja kub, Looks like you are re-inventing wheel here. All you are looking for is a fault tolerance system, you should consider exploring hystrix - latency and fault tolerance for distributed systems library.
https://github.com/Netflix/Hystrix -Ravi ________________________________________ From: Ja kub [jjaku...@gmail.com] Sent: Thursday, June 13, 2013 1:11 AM To: Tomcat Users List Subject: Re: http request (no only session) replication in cluster Christopher Thx for response, I will inform guys from business about what You have written, and let them consider it Regards Jakub On Wed, Jun 12, 2013 at 4:10 PM, Christopher Schultz < ch...@christopherschultz.net> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Jakob, > > On 6/11/13 5:04 PM, Ja kub wrote: > > requirement is system should be possible to process 160 req/sec > > (200 is better to multiply) and system is kind of failover proxy > > itself > > > > there are 2 backing webservices, each can answer max 20s, it there > > is timeout on first, I must call the second, if there is timeout on > > second I send soap fault to client, so usually it shouldn't be more > > than 20s per req, guys say that normally it is 7-10 > > seconds/request, but in worst case it is 2*20s*160 requests/s ~= > > 6400 pending requests (and according to deal we must fulfill worst > > case) > > If you have 2 member nodes and one of them starts to slow down, then > you'll see pretty much all requests re-tried on the second node, which > will slow down that one. I think you'll end up seeing a storm of > requests bouncing back and forth. > > Worse, the initial request will continue processing on the 1st node, > ignorant of the fact that the lb has given up and tried the other > node. It's just going to fall apart from there. > > Honestly, this should be able to be handled at your lb -- can't you > set a time-out there? > > > even if there are so many requests they are pending on sockets, I > > try to do it with NIO, asynchronous servlets and async cxf - both > > async cxf webservice is exposed by me, and I also call backing ws > > with async cxf I think even one tomcat on one server should be able > > to serve such 6400 pending requests with 160req/s, apart from proxy > > there are also 4-6 inserts into database (cli req, resp; 1st ws > > call, resp; 2nd ws call, resp > > > > how do You assess such architecture/attitude ? do You expect > > problems with async exposed webservice based on async servlet and > > NIO, and async cxf ws client ? afaik cxf use thread locals, are > > they all right with tomcat async servlets ? (I don't define > > threadlocals by myself, only cxf possibly does) > > It's not a socket-resource issue, it's a raw work-load issue: you have > a large amount of load and it looks like you can't handle it very > well. I would recommend more nodes, first, and then seriously consider > whether re-trying on a second node is appropriate if the first node > takes too long. > > What you should probably do is actually profile your code to find out > what is taking so long. Using tricks like ThreadLocals can shed > microseconds off of a request, not whole seconds. > > You might want to consider whether you can do less work during a > request -- perhaps split a single transaction into more than one. Or, > just acknowledge that sometimes a transaction can take 10-20 seconds > (or 50?) and manage the clients' expectations. > > You also need to find out where your bottleneck is: RDBMSs, slow > disks, slow network links, etc. can all be much more significant than > things like software stack and exact implementation of your code. If > you are missing an index on a relational table, transactions that > should take a second or two can take tens of seconds. > > Start there: profile your application, find out what is slow, and fix > that. Don't try to work-around the problem with surprising > transactional re-tries, because they likely won't work the way you > hoped. Hey, once you fix your performance problem, perhaps you won't > need additional hardware. Also, your users will be very happy to see a > speed improvement. > > - -chris > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) > Comment: GPGTools - http://gpgtools.org > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iQIcBAEBCAAGBQJRuIFyAAoJEBzwKT+lPKRYr+4QAL7UCQvZ/CBIueIqhFDLkZ57 > v0uWcuukEtT8/8dNUBW2SGSE4OwDyH41Nsx7ZVo4W+lTnzduVbjXSvU4lXNDiY19 > 9MgpwuxZWlUxHAgOQ5NODIFwrHQK2GYMe8+Qo8OBVf6lOhVdB4PS/7XM7lrsnWBn > rpojl4rPm7+esciZPB1q15+PxCbgh4uGsI4KCXyZiW/gz/dLC2v8u6QiYfqoXgov > iutYtII+7f1E6I+Ag3LjmQwrzY7pRpHrotcJ4aCpyOs9EHTavKf3mapwY2HiOP+t > G9qwGuq5tUJhkBzF5Vdvqf+lCbdJHkQtLW3Z4vL4/XTK7SVSvjipFhsttZF4TII6 > 6QVQmjCJZRYdPDegzB+NVaCxPkdZLLdwHNFFfsZGabdTQDkAKOEXQiYjBqJ9n5nX > WRHvYLQtyGEj1e+0zqwCihRHie2TbfwdggtCoVaOF+8Zpguv3K9VRHwvFA/miA1i > JkYCfxKjyF/RoCyB4wZqCi5VsJjztQpq6uDQiUG0CACY1491sB35M+Vkqm3jqRbh > 0HXs1ckqZsw+2Y013kpCVs0eipOst5GD6XqXr6LTT/fQwEYWa3uVTk3/h2xDd9BT > DlTZrs1CNhqMBjNqUDUFkiiempf9kFkQhrao50CAilix95/VhdWkDjFcFSKKQ0/J > EkcONNIioMTN7cWzKNHf > =miI6 > -----END PGP SIGNATURE----- > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
