Re: Too many sockets being created. Bad architecture?
My question is : What is the standard way to architect web services which have a very long blocking synchronous operations ? Don't. Design an asynchronous exchange instead. Anne On 8/2/06, kk kk [EMAIL PROTECTED] wrote: Hi, I am developing a WS which provides an operation that takes around 25 seconds to complete. I can not speed this up nor I can not use a polling pattern or a callback to inform the client of the result of the request. I want to make the WS server capable to handle 100 new requests per second, so 100 x 25 = 2500 sockets connections will exist at any point in time with my current design !! I am sure this is bad but am not sure what is the best way to go to get a scalable solution with a single server process. I am using Axis 1.2 and Tomcat on the server which is a Sun Solaris box. My test client is a Axis/Java application though I need to be able to support other clients too. As expected, I can see huge numbers of open sockets and FDs using unix commands like netstat and lsof. I have been experimenting with TCP/IP kernel setting on the server and using ideas from http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic=/com.ibm.websphere.express.doc/info/exp/ae/tprf_tunesolaris.html I have used ndd -set /dev/tcp tcp_conn_req_max_q 8000 to increase number of sockets that the server can handle and this does seem to allow a 1500+ parallel sockets to exist. My question is : What is the standard way to architect web services which have a very long blocking synchronous operations ? Assuming I have a small number of clients each making many WS calls is there a way to somehow multiplex many SOAP requests for eackh client down one socket? Is there a way to get Axis/Tomcat server to use UDP/IP rather than TCP/IP? I am new to this and all ideas much appreciated. Cheers, KP ___ All new Yahoo! Mail The new Interface is stunning in its simplicity and ease of use. - PC Magazine http://uk.docs.yahoo.com/nowyoucan.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Too many sockets being created. Bad architecture?
I would second that... asynchronous is definitely the way to go. And I don't think it's a just a matter of the number of sockets at the OS level, it's also a matter of the number of threads your web/app server can run. A heavy-duty app server like weblogic has 15 execute threads by default. Here you're talking about a 100 new requests coming in per second, and each taking 25 seconds to complete. So even if you had 10 weblogic servers in a load-balanced cluster, you would still run out of available threads after just a few seconds ... Regards, Sunil D'Monte Tavant Technologies http://www.tavant.com -Original Message- From: Anne Thomas Manes [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 02, 2006 17:18 To: axis-user@ws.apache.org Subject: Re: Too many sockets being created. Bad architecture? My question is : What is the standard way to architect web services which have a very long blocking synchronous operations ? Don't. Design an asynchronous exchange instead. Anne On 8/2/06, kk kk [EMAIL PROTECTED] wrote: Hi, I am developing a WS which provides an operation that takes around 25 seconds to complete. I can not speed this up nor I can not use a polling pattern or a callback to inform the client of the result of the request. I want to make the WS server capable to handle 100 new requests per second, so 100 x 25 = 2500 sockets connections will exist at any point in time with my current design !! I am sure this is bad but am not sure what is the best way to go to get a scalable solution with a single server process. I am using Axis 1.2 and Tomcat on the server which is a Sun Solaris box. My test client is a Axis/Java application though I need to be able to support other clients too. As expected, I can see huge numbers of open sockets and FDs using unix commands like netstat and lsof. I have been experimenting with TCP/IP kernel setting on the server and using ideas from http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic= /com.ibm.websphere.express.doc/info/exp/ae/tprf_tunesolaris.html I have used ndd -set /dev/tcp tcp_conn_req_max_q 8000 to increase number of sockets that the server can handle and this does seem to allow a 1500+ parallel sockets to exist. My question is : What is the standard way to architect web services which have a very long blocking synchronous operations ? Assuming I have a small number of clients each making many WS calls is there a way to somehow multiplex many SOAP requests for eackh client down one socket? Is there a way to get Axis/Tomcat server to use UDP/IP rather than TCP/IP? I am new to this and all ideas much appreciated. Cheers, KP ___ All new Yahoo! Mail The new Interface is stunning in its simplicity and ease of use. - PC Magazine http://uk.docs.yahoo.com/nowyoucan.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Too many sockets being created. Bad architecture?
Hi Sunil, Anne et al,I am not designing this system from scratch and the SOAP API with this 25 sec blocking synchronous operation is a requirement. (By "asynchronous exchange" I take it Anne means mean get my server to do a 'WS callback' ... unfortunately that is a not allowed API change.)My server is using Tomcat 5.5 within JBoss and by default Tomcat is using 250 threads in its pool; I have upped this to 2500, have modified Tomcat to use native not Java code for handling the HTTP requests (See http://tomcat.apache.org/tomcat-5.5-doc/apr.html ) have made TCP/IP stack server modifications.This is giving me around a max of 60 to 80 WS requests per second each lasting 25 seconds with very low CPU utilization on the server.This is running stable in overnight tests with around 1300 Java threads active in the server (mostly Tomcat threads) however I am not sure how to push this rate up further or even if I am going down the right road here.So that is why I am wondering if at a TCP/IP level is there is anything I can do to make communication more efficient e.g. reusing same socket for many and overlapping SOAP requests responses or maybe UDP not TCP.I thought this would be a common problem but it v hard to find info on the web on this.Thanks,KPSunil D'Monte [EMAIL PROTECTED] wrote: I would second that... asynchronous is definitely the way to go. And Idon't think it's a just a matter of the number of sockets at the OSlevel, it's also a matter of the number of threads your web/app servercan run. A heavy-duty app server like weblogic has 15 execute threads bydefault. Here you're talking about a 100 new requests coming in persecond, and each taking 25 seconds to complete. So even if you had 10weblogic servers in a load-balanced cluster, you would still run out ofavailable threads after just a few seconds ...Regards,Sunil D'MonteTavant Technologieshttp://www.tavant.com -Original Message- From: Anne Thomas Manes [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 02, 2006 17:18 To: axis-user@ws.apache.org Subject: Re: Too many sockets being created. Bad architecture? My question is : What is the standard way to architect web services which have a very long blocking synchronous operations ? Don't. Design an asynchronous exchange instead. Anne On 8/2/06, kk kk <[EMAIL PROTECTED]> wrote: Hi, I am developing a WS which provides an operation that takes around 25 seconds to complete. I can not speed this up nor I can not use a polling pattern or a callback to inform the client of the result of the request. I want to make the WS server capable to handle 100 new requests per second, so 100 x 25 = 2500 sockets connections will exist at any point in time with my current design !! I am sure this is bad but am not sure what is the best way to go to get a scalable solution with a single server process. I am using Axis 1.2 and Tomcat on the server which is a Sun Solaris box. My test client is a Axis/Java application though I need to be able to support other clients too. As expected, I can see huge numbers of open sockets and FDs using unix commands like netstat and lsof. I have been experimenting with TCP/IP kernel setting on the server and using ideas fromhttp://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic= /com.ibm.websphere.express.doc/info/exp/ae/tprf_tunesolaris.html I have used ndd -set /dev/tcp tcp_conn_req_max_q 8000 to increase number of sockets that the server can handle and this does seem to allow a 1500+ parallel sockets to exist. My question is : What is the standard way to architect web services which have a very long blocking synchronous operations ? Assuming I have a small number of clients each making many WS calls is there a way to somehow multiplex many SOAP requests for eackh client down one socket? Is there a way to get Axis/Tomcat server to use UDP/IP rather than TCP/IP? I am new to this and all ideas much appreciated. Cheers, KP___ All new Yahoo! Mail "The new Interface is stunning in its simplicity and ease of use." - PC Magazine http://uk.docs.yahoo.com/nowyoucan.html- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -To unsubscribe, e-mail: [EMAIL PROTECTED]For additional commands, e-mail: [EMAIL PROTECTED] All new Yahoo! Mail "The new Interface is stunning in its simplicity and ease of use." - PC Magazine
Re: Too many sockets being created. Bad architecture?
Sign up for one of those Niagara server (T2000) and run an evaluation on them. You will need really good servers and as much network capacity to match in an intranet environment to get this kind of performance. Search for the keyword C10K and read up more :) thanks, dims On 8/2/06, Kiggy Pop [EMAIL PROTECTED] wrote: Hi Sunil, Anne et al, I am not designing this system from scratch and the SOAP API with this 25 sec blocking synchronous operation is a requirement. (By asynchronous exchange I take it Anne means mean get my server to do a 'WS callback' ... unfortunately that is a not allowed API change.) My server is using Tomcat 5.5 within JBoss and by default Tomcat is using 250 threads in its pool; I have upped this to 2500, have modified Tomcat to use native not Java code for handling the HTTP requests (See http://tomcat.apache.org/tomcat-5.5-doc/apr.html ) have made TCP/IP stack server modifications. This is giving me around a max of 60 to 80 WS requests per second each lasting 25 seconds with very low CPU utilization on the server. This is running stable in overnight tests with around 1300 Java threads active in the server (mostly Tomcat threads) however I am not sure how to push this rate up further or even if I am going down the right road here. So that is why I am wondering if at a TCP/IP level is there is anything I can do to make communication more efficient e.g. reusing same socket for many and overlapping SOAP requests responses or maybe UDP not TCP. I thought this would be a common problem but it v hard to find info on the web on this. Thanks, KP Sunil D'Monte [EMAIL PROTECTED] wrote: I would second that... asynchronous is definitely the way to go. And I don't think it's a just a matter of the number of sockets at the OS level, it's also a matter of the number of threads your web/app server can run. A heavy-duty app server like weblogic has 15 execute threads by default. Here you're talking about a 100 new requests coming in per second, and each taking 25 seconds to complete. So even if you had 10 weblogic servers in a load-balanced cluster, you would still run out of available threads after just a few seconds ... Regards, Sunil D'Monte Tavant Technologies http://www.tavant.com -Original Message- From: Anne Thomas Manes [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 02, 2006 17:18 To: axis-user@ws.apache.org Subject: Re: Too many sockets being created. Bad architecture? My question is : What is the standard way to architect web services which have a very long blocking synchronous operations ? Don't. Design an asynchronous exchange instead. Anne On 8/2/06, kk kk wrote: Hi, I am developing a WS which provides an operation that takes around 25 seconds to complete. I can not speed this up nor I can not use a polling pattern or a callback to inform the client of the result of the request. I want to make the WS server capable to handle 100 new requests per second, so 100 x 25 = 2500 sockets connections will exist at any point in time with my current design !! I am sure this is bad but am not sure what is the best way to go to get a scalable solution with a single server process. I am using Axis 1.2 and Tomcat on the server which is a Sun Solaris box. My test client is a Axis/Java application though I need to be able to support other clients too. As expected, I can see huge numbers of open sockets and FDs using unix commands like netstat and lsof. I have been experimenting with TCP/IP kernel setting on the server and using ideas from http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic= /com.ibm.websphere.express.doc/info/exp/ae/tprf_tunesolaris.html I have used ndd -set /dev/tcp tcp_conn_req_max_q 8000 to increase number of sockets that the server can handle and this does seem to allow a 1500+ parallel sockets to exist. My question is : What is the standard way to architect web services which have a very long blocking synchronous operations ? Assuming I have a small number of clients each making many WS calls is there a way to somehow multiplex many SOAP requests for eackh client down one socket? Is there a way to get Axis/Tomcat server to use UDP/IP rather than TCP/IP? I am new to this and all ideas much appreciated. Cheers, KP ___ All new Yahoo! Mail The new Interface is stunning in its simplicity and ease of use. - PC Magazine http://uk.docs.yahoo.com/nowyoucan.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED
Re: Too many sockets being created. Bad architecture?
If there is another box available you could try a scheme like the following:: Use the other box as a proxy. It presents a synchronous web service to clients, which presumably connect via http. It passes the requests and responses to/from the box that implements the service (doing the heavy lifting) in an asynchronous pattern. It is thus called back by that service with the response to each request. It could use JMS for this purpose, for instance. Each request/response to the implementation box would be correlated with the client http connection on which the response was to be returned to the client. The proxy box would need 2500 or so open sockets (plus a small number for traffic to/from the implementation box) but would not need 2500 threads (unless the proxy used the thread-per-request model that Axis uses). Essentially, the proxy box is bridging between synchronous http and a message queueing system. The implementation box would need just a small number of open sockets and could use any desired threading architecture, such as some kind of thread pool. In general, the synchronization overhead for thousands of threads is a signficant performance hit even if the threads are blocking most of the time. There are efficient architectures for handling many sockets. Google for the Reactor pattern, or consult the Patterns of Software Architecture book (vol 2) or the papers of Doug Schmidt and the ACE group http://www.cs.wustl.edu/~schmidt/ACE.html . Jeff On 8/2/06, Kiggy Pop [EMAIL PROTECTED] wrote: Hi Sunil, Anne et al, I am not designing this system from scratch and the SOAP API with this 25 sec blocking synchronous operation is a requirement. (By asynchronous exchange I take it Anne means mean get my server to do a 'WS callback' ... unfortunately that is a not allowed API change.) My server is using Tomcat 5.5 within JBoss and by default Tomcat is using 250 threads in its pool; I have upped this to 2500, have modified Tomcat to use native not Java code for handling the HTTP requests (See http://tomcat.apache.org/tomcat-5.5-doc/apr.html ) have made TCP/IP stack server modifications. This is giving me around a max of 60 to 80 WS requests per second each lasting 25 seconds with very low CPU utilization on the server. This is running stable in overnight tests with around 1300 Java threads active in the server (mostly Tomcat threads) however I am not sure how to push this rate up further or even if I am going down the right road here. So that is why I am wondering if at a TCP/IP level is there is anything I can do to make communication more efficient e.g. reusing same socket for many and overlapping SOAP requests responses or maybe UDP not TCP. I thought this would be a common problem but it v hard to find info on the web on this. Thanks, KP Sunil D'Monte [EMAIL PROTECTED] wrote: I would second that... asynchronous is definitely the way to go. And I don't think it's a just a matter of the number of sockets at the OS level, it's also a matter of the number of threads your web/app server can run. A heavy-duty app server like weblogic has 15 execute threads by default. Here you're talking about a 100 new requests coming in per second, and each taking 25 seconds to complete. So even if you had 10 weblogic servers in a load-balanced cluster, you would still run out of available threads after just a few seconds ... Regards, Sunil D'Monte Tavant Technologies http://www.tavant.com -Original Message- From: Anne Thomas Manes [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 02, 2006 17:18 To: axis-user@ws.apache.org Subject: Re: Too many sockets being created. Bad architecture? My question is : What is the standard way to architect web services which have a very long blocking synchronous operations ? Don't. Design an asynchronous exchange instead. Anne On 8/2/06, kk kk wrote: Hi, I am developing a WS which provides an operation that takes around 25 seconds to complete. I can not speed this up nor I can not use a polling pattern or a callback to inform the client of the result of the request. I want to make the WS server capable to handle 100 new requests per second, so 100 x 25 = 2500 sockets connections will exist at any point in time with my current design !! I am sure this is bad but am not sure what is the best way to go to get a scalable solution with a single server process. I am using Axis 1.2 and Tomcat on the server which is a Sun Solaris box. My test client is a Axis/Java application though I need to be able to support other clients too. As expected, I can see huge numbers of open sockets and FDs using unix commands like netstat and lsof. I have been experimenting with TCP/IP kernel setting on the server and using ideas from http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic= /com.ibm.websphere.express.doc/info/exp/ae/tprf_tunesolaris.html I have used ndd -set /dev/tcp
