Chris, On single tomcat instance how to map each app to different port number?
On Sun, 31 May 2020, 15:44 Christopher Schultz, < ch...@christopherschultz.net> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Ayub, > > On 5/29/20 20:23, Ayub Khan wrote: > > Chris, > > > > You might want (2) and (3) to have their own, independent > > connector and thread pool, just to be safe. You don't want a > > connection in (1) to stall because a loopback connection can't be > > made to (2)/(3). Meanwhile, it's sitting there making no progress > > but also consuming a connection+thread. > > > > *there is only one connector per tomcat where all the applications > receive > > the requests, they do not have independent connector and thread > > pool per tomcat. How to configure independent connector and thread > > pool per application per tomcat instance ? below is the current > > connector > config in > > each tomcat instance:* > > You can't allocate a connector to a particular web application -- at > least not in the way that you think. > > What you have to do if use different port numbers. Users will never > use them, though. But since you have nginx (finally! A reason to have > it!), you can map /app1 to port 8080 and /app2 to port 8081 and /app3 > to port 8083 or whatever you want. > > Internal loopback connections will either have to go through nginx > (which I wouldn't recommend) or know the correct port numbers to use > (which I *do* recommend). > > - -chris > > > *<Connector port="8080" > > protocol="org.apache.coyote.http11.Http11NioProtocol" > > connectionTimeout="20000" URIEncoding="UTF-8" > > redirectPort="8443" />* > > > > > > > > On Fri, May 29, 2020 at 9:05 PM Christopher Schultz < > > ch...@christopherschultz.net> wrote: > > > > Ayub, > > > > On 5/28/20 17:25, Ayub Khan wrote: > >>>> Nginx is being used for image caching and converting https to > >>>> http requests before hitting tomcat. > > So you encrypt between the ALB and your app server nodes? That's > > fine, though nginx probably won't offer any performance improvement > > for images (unless it's really caching dynamically-generated images > > from your application) or TLS termination. > > > >>>> The behavior I am noticing is application first throws > >>>> Borken pipe client abort exception at random apis calls > >>>> followed by socket timeout and then database connection leak > >>>> errors. This happens only during high load. > > > > If you are leaking connections, that's going to be an application > > resource-management problem. Definitely solve that, but it > > shouldn't affect anything with Tomcat connections and/or threads. > > > >>>> During normal traffic open files for tomcat process goes up > >>>> and down to not more than 500. However during high traffic if > >>>> I keep track of the open files for each tomcat process as > >>>> soon as the open files count reaches above 10k that tomcat > >>>> instance stops serving the requests. > > > > Any other errors shown in the logs? Like OutOfMemoryError (for > > e.g. open files)? > > > >>>> If the open file count goes beyond 5k its sure that this > >>>> number will never come back to below 500 at this point we > >>>> need to restart tomcat. > >>>> > >>>> > >>>> There are three application installed on each tomcat > >>>> instance, > >>>> > >>>> 1) portal: portal calls (2) and (3) using localhost, should > >>>> we change this to use dns name instead of localhost calls ? > >>>> > >>>> 2) Services for portal 3) Services for portal and mobile > >>>> clients > > > > Are they all sharing the same connector / thread pool? > > > > You might want (2) and (3) to have their own, independent > > connector and thread pool, just to be safe. You don't want a > > connection in (1) to stall because a loopback connection can't be > > made to (2)/(3). Meanwhile, it's sitting there making no progress > > but also consuming a connection+thread. > > > > -chris > > > >>>> On Thu, May 28, 2020 at 4:50 PM Christopher Schultz < > >>>> ch...@christopherschultz.net> wrote: > >>>> > >>>> Ayub, > >>>> > >>>> On 5/27/20 19:43, Ayub Khan wrote: > >>>>>>> If we have 18 core CPU and 100GB RAM. What value can I > >>>>>>> set for maxConnections ? > >>>> Your CPU and RAM really have nothing to do with it. It's more > >>>> about your usage profile. > >>>> > >>>> For example, if you are serving small static files, you can > >>>> serve a million requests a minute on a Raspberry Pi, many of > >>>> them concurrently. > >>>> > >>>> But if you are performing fluid dynamic simulations with > >>>> each request, you will obviously need more horsepower to > >>>> service a single request, let alone thousands of concurrent > >>>> requests. > >>>> > >>>> If you have tons of CPU and memory to spare, feel free to > >>>> crank-up the max connections. The default is 10000 which is > >>>> fairly high. At some point, you will run out of connection > >>>> allocation space in the OS's TCP/IP stack, so that is really > >>>> your upper-limit. You simply cannot have more than the OS > >>>> will allow. See https://stackoverflow.com/a/2332756/276232 > >>>> for some information about that. > >>>> > >>>> Once you adjust your settings, perform a load-test. You may > >>>> find that adding more resources actually slows things down. > >>>> > >>>>>>> Want to make sure we are utilizing the hardware to the > >>>>>>> max capacity. Is there any config of tomcat which > >>>>>>> enabled could help serve more requests per tomcat > >>>>>>> instance. > >>>> > >>>> Not really. Improving performance usually come down to tuning > >>>> the application to make the requests take less time to > >>>> process. Tomcat is rarely the source of performance problems > >>>> (but /sometimes/ is, and it's usually a bug that can be > >>>> fixed). > >>>> > >>>> You can improve throughput somewhat by pipelineing requests. > >>>> That means HTTP keepalive for direct connections (but with a > >>>> small timeout; you don't want clients who aren't making any > >>>> follow-up requests to waste your resources waiting for a > >>>> keep-alive timeout to close a connection). For proxy > >>>> connections (e.g. from nginx), you'll want those connections > >>>> to remain open as long as possible to avoid the > >>>> re-negotiation of TCP and possibly TLS handshakes. Using > >>>> HTTP/2 can be helpful for performance, at the cost of some > >>>> CPU on the back-end to perform the complicated connection > >>>> management that h2 requires. > >>>> > >>>> Eliminating useless buffering is often very helpful. That's > >>>> why I asked about nginx. What are you using it for, other > >>>> than as a barrier between the load-balancer and your Tomcat > >>>> instances? If you remove nginx, I suspect you'll see a > >>>> measurable performance increase. This isn't a knock against > >>>> nginx: you'd see a performance improvement by removing *any* > >>>> reverse-proxy that isn't providing any value. But you haven't > >>>> said anything about why it's there in the first place, so I > >>>> don't know if it /is/ providing any value to you. > >>>> > >>>>>>> The current setup is able to handle most of the load, > >>>>>>> however there are predictable times where there is an > >>>>>>> avalanche of requests and thinking how to handle it > >>>>>>> gracefully. > >>>> > >>>> You are using AWS: use auto-scaling. That's what it's for. > >>>> > >>>> -chris > >>>> > >>>>>>> On Wed, May 27, 2020 at 5:38 PM Christopher Schultz < > >>>>>>> ch...@christopherschultz.net> wrote: > >>>>>>> > >>>>>>> Ayub, > >>>>>>> > >>>>>>> On 5/27/20 09:26, Ayub Khan wrote: > >>>>>>>>>> previously I was using HTTP/1.1 connector, > >>>>>>>>>> recently I changed to NIO2 to see the > >>>>>>>>>> performance. I read that NIO2 is non blocking so > >>>>>>>>>> trying to check how this works. > >>>>>>> > >>>>>>> Both NIO and NIO2 are non-blocking. They use different > >>>>>>> strategies for certain things. Anything but the "BIO" > >>>>>>> connector will be non-blocking for most operations. > >>>>>>> The default is NIO. > >>>>>>> > >>>>>>>>>> which connector protocol do you recommend and > >>>>>>>>>> best configuration for the connector ? > >>>>>>> This depends on your environment, usage profile, etc. > >>>>>>> Note that non-blocking IO means more CPU usage: there > >>>>>>> is a trade-off. > >>>>>>> > >>>>>>>>>> Which stable version of tomcat would you > >>>>>>>>>> recommend ? > >>>>>>> > >>>>>>> Always the latest, of course. Tomcat 8.0 is > >>>>>>> unsupported, replaced by Tomcat 8.5. Tomcat 9.0 is > >>>>>>> stable and probably the best version if you are looking > >>>>>>> to upgrade. Both Tomcat 8.5 and 9.0 are continuing to > >>>>>>> get regular updates. But definitely move away from > >>>>>>> 8.0. > >>>>>>> > >>>>>>>>>> Are there any ubuntu specific configs for tomcat > >>>>>>>>>> ? > >>>>>>> No. There is nothing particular special about Ubuntu. > >>>>>>> Linux is one of the most well-performing platforms for > >>>>>>> the JVM. I wouldn't recommend switching platforms. > >>>>>>> > >>>>>>> Why are you using nginx? You already have > >>>>>>> load-balancing happening in the ALB. Inserting another > >>>>>>> layer of proxying is probably just adding another > >>>>>>> buffer to the mix. I'd remove nginx if it's not > >>>>>>> providing any specific, measurable benefit. > >>>>>>> > >>>>>>>>>> We are using OkHttp client library to call rest > >>>>>>>>>> api and stack trace shows failure at the api > >>>>>>>>>> call. The api being called is running on the same > >>>>>>>>>> tomcat instance (different context) usring url > >>>>>>>>>> localhost. This does not happen when number of > >>>>>>>>>> requests is less. > >>>>>>> > >>>>>>> Your Tomcat server is calling this REST API? Or your > >>>>>>> server is serving those API requests? If your service > >>>>>>> is calling itself, then you have to make sure you have > >>>>>>> double-capacity: every incoming request will cause a > >>>>>>> loopback request to your own service. > >>>>>>> > >>>>>>> Other than the timeouts, are you able to handle the > >>>>>>> load with your existing infrastructure? Sometimes, the > >>>>>>> solution is simply to throw most hardware at the > >>>>>>> problem. > >>>>>>> > >>>>>>> -chris > >>>>>>> > >>>>>>>>>> On Wed, May 27, 2020 at 11:48 AM Mark Thomas > >>>>>>>>>> <ma...@apache.org> wrote: > >>>>>>>>>> > >>>>>>>>>>> On 26/05/2020 23:28, Ayub Khan wrote: > >>>>>>>>>>>> Hi, > >>>>>>>>>>>> > >>>>>>>>>>>> During high load I am seeing below error on > >>>>>>>>>>>> tomcat logs > >>>>>>>>>>>> > >>>>>>>>>>>> java.util.concurrent.ExecutionException: > >>>>>>>>>>>> java.net > >>>>>>>>>>> .SocketTimeoutException: > >>>>>>>>>>>> timeout > >>>>>>>>>>> > >>>>>>>>>>> And the rest of that stack trace? It is hard > >>>>>>>>>>> to provide advice without context. We need to > >>>>>>>>>>> know what is timing out when trying to do > >>>>>>>>>>> what. > >>>>>>>>>>> > >>>>>>>>>>>> We have 4 C5.18x large vms running tomcat 8 > >>>>>>>>>>>> behind AWS application load balancer. We are > >>>>>>>>>>>> seeing socket timeouts during peak hours. > >>>>>>>>>>>> What should be the configuration of tomcat if > >>>>>>>>>>>> we get 60,000 to 70,000 requests per > >>>>>>>>>>> minute > >>>>>>>>>>>> on an average ? > >>>>>>>>>>>> > >>>>>>>>>>>> Tomcat 8.0.32 on Ubuntu 16.04.5 LTS > >>>>>>>>>>> > >>>>>>>>>>> Tomcat 8.0.x is no longer supported. > >>>>>>>>>>> > >>>>>>>>>>>> Below is the java version: > >>>>>>>>>>>> > >>>>>>>>>>>> java version "1.8.0_181" Java(TM) SE Runtime > >>>>>>>>>>>> Environment (build 1.8.0_181-b13) Java > >>>>>>>>>>>> HotSpot(TM) 64-Bit Server VM (build > >>>>>>>>>>>> 25.181-b13, mixed mode) > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Below is the server.xml connector > >>>>>>>>>>>> configuration: > >>>>>>>>>>>> > >>>>>>>>>>>> <Connector port="8080" > >>>>>>>>>>>> protocol="org.apache.coyote.http11.Http11Nio2Protocol" > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > > > >>>>>>>>>>>> > Why NIO2? > >>>>>>>>>>> > >>>>>>>>>>> Mark > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> connectionTimeout="20000" > >>>>>>>>>>>> > >>>>>>>>>>>> URIEncoding="UTF-8" redirectPort="8443" /> > >>>>>>>>>>>> > >>>>>>>>>>>> We have 4 C5.18x large vms and each vm has > >>>>>>>>>>>> nginx and tomcat instance running. All the 4 > >>>>>>>>>>>> vms are connected to AWS application load > >>>>>>>>>>>> balancer. > >>>>>>>>>>>> > >>>>>>>>>>>> I tried to add maxConnections=50000 but this > >>>>>>>>>>>> does not seem to have any affect and still > >>>>>>>>>>>> saw the timeouts > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks and Regards Ayub > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> ------------------------------------------------------------ > - --- > > > >>>>>>>>>>> > - --- > >>>> > >>>>>>>>>>> > >>>>>>>>>>> > > --- > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >>>>>>>>>>> For additional commands, e-mail: > >>>>>>>>>>> users-h...@tomcat.apache.org > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>>> --------------------------------------------------------------- > - --- > > > >>>>>>>> > - --- > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>> > >>>>>>>> > >>>>>>>> > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >>>>>>>> For additional commands, e-mail: > >>>>>>>> users-h...@tomcat.apache.org > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>>>> ------------------------------------------------------------------ > - --- > >>>>> > >>>>> > >>>>> > > > >>>>> > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >>>>> For additional commands, e-mail: > >>>>> users-h...@tomcat.apache.org > >>>>> > >>>>> > >>>> > >>>> -- > >>>> > >> > >> --------------------------------------------------------------------- > >> > >> > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >> For additional commands, e-mail: users-h...@tomcat.apache.org > >> > >> > > > -----BEGIN PGP SIGNATURE----- > Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ > > iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl7TppsACgkQHPApP6U8 > pFhQgBAAshTFIepyQLQ05z5+ri+Btx3eWLwnrawkLosXvou68u17XPZPi1LwVHCL > 1hqxf2dZZOEwu1Lks4P1Y8SZTNE/RzzKSIkJ/edQUA7Aqfz/kl+rYPmft0EmFO5W > uwaDH9XbOa3B2s+iTLWoIjEWForZJl7y9vUYr8Xug85DQXUKTUujGqqOVw6xezo6 > HkoR+940xxMj3Cz3q9aI9mYYI9Fj/GVMXro476cm6eqkF37vzviXxKINTBhdcJ2K > h8e/kWCNNMY4E4j5otqkl8LUbHV0LUh4IUOl30h9ibpwlcmRMWEh1N+KQS0zDfpK > c6eqlHBmI3KnJ9aphlMHf6regiydT1phLnLWKwMlxf/l1N7KqGCIUbYPVmfOKn6I > uWBNmfOmNL1svj3V1eimSYmmDvRNCLGvjI8BF/qxDv0RAEE/WkswbzEy47GD1usx > vweeHHQ4OVMbiwtQ3dgbcOyS+7Gck5k81Ul+CLr8QJSozYzChe6gjh7RaPzzbIRk > a9qtB0PvweDBmws4NXG5hdU/P44cb3lGkIPtSgDdzo+5SvEgSme8pkQzyxWahAfi > tBqyGmdWuUjZEpgT8YEg5nisHXws0Xdsj6IGpTRl3yBIv5k5lZtQTc4vGBMfwRDz > ypr8kIZIfSInBs63aHHycFIiS6uTGrOHw9oobXjidR/EVNJ5YsI= > =OhWG > -----END PGP SIGNATURE----- > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
