Re: Tomcat 9.0.0 multiple thread issue
Sorry for late reply. We have fixed the issue. Kubernetes ingress file was causing multiple requests to be sent over and over again. Proxy timeout along with read and send timeouts were not present in that file. We have added them and now the process is working fine allowing it to complete with only one single thread. Thank you all for your support. Regards, Ripu daman On Tue 9 Jun, 2020, 10:20 PM Christopher Schultz, < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Ripu, > > On 6/9/20 08:03, Ripu Daman wrote: > > Hello everyone, > > > > I work in an IT firm as a software engineer. I don't know whether > > it's a right platform to ask questions based on personal projects > > but i believe i will get some help from here. > > > > We are facing an issue regarding multiple threads being created for > > a servlet execution running in tomcat 9.0.0 > > > > Servlet's job is to publish customer leads to salesforce via soap > > protocol. The logic has been designed in such a way that in a > > single thread it can publish multiple leads to salesforce one by > > one. However we recently migrated our java application to > > kubernetes adopting containerized approach. Post this we are facing > > an issue in which, when the process starts publishing leads to > > salesforce one by one, after a few uploads a new thread gets > > created and it starts the whole process from beginning and then > > after a few seconds another thread kicks off repeating the whole > > process. It keeps going on until session timeout exceeds and we get > > a 504 error. But the upload process keeps going on in the > > background and stops after 2-3 threads. > > Where is your queue stored? > > > Before moving to kubernetes, the process was working fine. In order > > to serve multiple requests at the same time, servlet is not > > designed to be thread safe. > > It's NOT designed to be thread-safe? I hope you meant it IS designed > to be thread-safe. > > > But now we are facing this issue and unable to identify the root > > cause. > > > > Here's a sample of threads from the logs : [http-nio-8080-exec-9] > > INFO [http-nio-8080-exec-9] INFO [http-nio-8080-exec-9] INFO > > [http-nio-8080-exec-7] INFO [http-nio-8080-exec-7] INFO > > [http-nio-8080-exec-9] INFO [http-nio-8080-exec-9] INFO > > > > [http-nio-8080-exec-7] INFO [http-nio-8080-exec-7] INFO > > [http-nio-8080-exec-7] INFO [http-nio-8080-exec-3] INFO > > [http-nio-8080-exec-7] INFO [http-nio-8080-exec-3] INFO > > These are request-processing threads, so something is making HTTP > requests to Tomcat to do that work. > > - -chris > -BEGIN PGP SIGNATURE- > Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ > > iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl7fvbEACgkQHPApP6U8 > pFi9Uw/+O9Be+Bp7n8co6Rz7gkvtXZbKT5KlxpWtT3gVNluN4Lt824Hgb5C7wmMd > AjZfvFO0yn1rIJsY+G0r/25lsdIVLZZW3ehM28eRgc+Up3sI66w8zvnOGhly7E6f > CKvSKkINLrS9+BQ0HrLkqHxn+U4y2Ts8yfRZOelC8dGtFfj+U3cSsc75LGMUiREM > SlYabv4dDzemfBqG/08zXfQ+laQStg9Cf+rY//ZZGnLWR74obJJB/rlhrNMAzND0 > qR5Z0LebYXGQEt6nElYtDtAosJlnlpX30fLX6aVwwIXfgPqeIgBF9PskIfwx1NU5 > sF3nIHO4wE+KsMqM22fqMOU6keBR/Ac8ktMEws/d6PEL1oXUhcsXMTulh6RrxHWA > ImTlksYyIp54flIw3cwiooVW1VrgYQisIIucsWPzLSw7QsE9CyZADqYYm6N7Eo60 > VpdHAOLTX4zfJoMlf7NWuMdK8dFI8wkLUmCURxYkWysWx+MJ55BDgDy9kTw3QZoa > lBZs9bPirGU8r7qmjbrj5L/Q0lpeqQSSj9ixBzAJKGBAYzYJHKWPpOcI2NW1hn6F > vUi01xB67zXboH1EcLVHqMc9srA5UjaDKzwnCcwde1kBaxRbwHFIOq1s+BflXdEc > 69gaaPqifMnp1pLaJoonPDyZ3QszSAm5SiUi6lKmofF0UBqTzYs= > =SxID > -END PGP SIGNATURE- > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Tomcat 9.0.0 multiple thread issue
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Ripu, On 6/9/20 08:03, Ripu Daman wrote: > Hello everyone, > > I work in an IT firm as a software engineer. I don't know whether > it's a right platform to ask questions based on personal projects > but i believe i will get some help from here. > > We are facing an issue regarding multiple threads being created for > a servlet execution running in tomcat 9.0.0 > > Servlet's job is to publish customer leads to salesforce via soap > protocol. The logic has been designed in such a way that in a > single thread it can publish multiple leads to salesforce one by > one. However we recently migrated our java application to > kubernetes adopting containerized approach. Post this we are facing > an issue in which, when the process starts publishing leads to > salesforce one by one, after a few uploads a new thread gets > created and it starts the whole process from beginning and then > after a few seconds another thread kicks off repeating the whole > process. It keeps going on until session timeout exceeds and we get > a 504 error. But the upload process keeps going on in the > background and stops after 2-3 threads. Where is your queue stored? > Before moving to kubernetes, the process was working fine. In order > to serve multiple requests at the same time, servlet is not > designed to be thread safe. It's NOT designed to be thread-safe? I hope you meant it IS designed to be thread-safe. > But now we are facing this issue and unable to identify the root > cause. > > Here's a sample of threads from the logs : [http-nio-8080-exec-9] > INFO [http-nio-8080-exec-9] INFO [http-nio-8080-exec-9] INFO > [http-nio-8080-exec-7] INFO [http-nio-8080-exec-7] INFO > [http-nio-8080-exec-9] INFO [http-nio-8080-exec-9] INFO > > [http-nio-8080-exec-7] INFO [http-nio-8080-exec-7] INFO > [http-nio-8080-exec-7] INFO [http-nio-8080-exec-3] INFO > [http-nio-8080-exec-7] INFO [http-nio-8080-exec-3] INFO These are request-processing threads, so something is making HTTP requests to Tomcat to do that work. - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl7fvbEACgkQHPApP6U8 pFi9Uw/+O9Be+Bp7n8co6Rz7gkvtXZbKT5KlxpWtT3gVNluN4Lt824Hgb5C7wmMd AjZfvFO0yn1rIJsY+G0r/25lsdIVLZZW3ehM28eRgc+Up3sI66w8zvnOGhly7E6f CKvSKkINLrS9+BQ0HrLkqHxn+U4y2Ts8yfRZOelC8dGtFfj+U3cSsc75LGMUiREM SlYabv4dDzemfBqG/08zXfQ+laQStg9Cf+rY//ZZGnLWR74obJJB/rlhrNMAzND0 qR5Z0LebYXGQEt6nElYtDtAosJlnlpX30fLX6aVwwIXfgPqeIgBF9PskIfwx1NU5 sF3nIHO4wE+KsMqM22fqMOU6keBR/Ac8ktMEws/d6PEL1oXUhcsXMTulh6RrxHWA ImTlksYyIp54flIw3cwiooVW1VrgYQisIIucsWPzLSw7QsE9CyZADqYYm6N7Eo60 VpdHAOLTX4zfJoMlf7NWuMdK8dFI8wkLUmCURxYkWysWx+MJ55BDgDy9kTw3QZoa lBZs9bPirGU8r7qmjbrj5L/Q0lpeqQSSj9ixBzAJKGBAYzYJHKWPpOcI2NW1hn6F vUi01xB67zXboH1EcLVHqMc9srA5UjaDKzwnCcwde1kBaxRbwHFIOq1s+BflXdEc 69gaaPqifMnp1pLaJoonPDyZ3QszSAm5SiUi6lKmofF0UBqTzYs= =SxID -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 9.0.0 multiple thread issue
I assume a servlet request is kicking off this process? If this is correct then are you 100% sure that something is not calling the servlet more than once? Perhaps a retry after a timeout or something? Turn your access log on and look at what requests are coming in to the server Paul On Tue, Jun 9, 2020 at 2:04 PM Ripu Daman wrote: > Hello everyone, > > I work in an IT firm as a software engineer. I don't know whether it's a > right platform to ask questions based on personal projects but i believe i > will get some help from here. > > We are facing an issue regarding multiple threads being created for a > servlet execution running in tomcat 9.0.0 > > Servlet's job is to publish customer leads to salesforce via soap protocol. > The logic has been designed in such a way that in a single thread it can > publish multiple leads to salesforce one by one. However we recently > migrated our java application to kubernetes adopting containerized > approach. > Post this we are facing an issue in which, when the process starts > publishing leads to salesforce one by one, after a few uploads a new thread > gets created and it starts the whole process from beginning and then after > a few seconds another thread kicks off repeating the whole process. It > keeps going on until session timeout exceeds and we get a 504 error. But > the upload process keeps going on in the background and stops after 2-3 > threads. > > Before moving to kubernetes, the process was working fine. In order to > serve multiple requests at the same time, servlet is not designed to be > thread safe. > > But now we are facing this issue and unable to identify the root cause. > > Here's a sample of threads from the logs : > [http-nio-8080-exec-9] INFO > [http-nio-8080-exec-9] INFO > [http-nio-8080-exec-9] INFO > [http-nio-8080-exec-7] INFO > [http-nio-8080-exec-7] INFO > [http-nio-8080-exec-9] INFO > [http-nio-8080-exec-9] INFO > > [http-nio-8080-exec-7] INFO > [http-nio-8080-exec-7] INFO > [http-nio-8080-exec-7] INFO > [http-nio-8080-exec-3] INFO > [http-nio-8080-exec-7] INFO > [http-nio-8080-exec-3] INFO > > The configurations for tomcat remains same as default. Session timeout is > set as 30 min (default). > > For more info please let me know. > > Hope will get some help from here. > > Thanks! > > regards, > Ripu daman >
Tomcat 9.0.0 multiple thread issue
Hello everyone, I work in an IT firm as a software engineer. I don't know whether it's a right platform to ask questions based on personal projects but i believe i will get some help from here. We are facing an issue regarding multiple threads being created for a servlet execution running in tomcat 9.0.0 Servlet's job is to publish customer leads to salesforce via soap protocol. The logic has been designed in such a way that in a single thread it can publish multiple leads to salesforce one by one. However we recently migrated our java application to kubernetes adopting containerized approach. Post this we are facing an issue in which, when the process starts publishing leads to salesforce one by one, after a few uploads a new thread gets created and it starts the whole process from beginning and then after a few seconds another thread kicks off repeating the whole process. It keeps going on until session timeout exceeds and we get a 504 error. But the upload process keeps going on in the background and stops after 2-3 threads. Before moving to kubernetes, the process was working fine. In order to serve multiple requests at the same time, servlet is not designed to be thread safe. But now we are facing this issue and unable to identify the root cause. Here's a sample of threads from the logs : [http-nio-8080-exec-9] INFO [http-nio-8080-exec-9] INFO [http-nio-8080-exec-9] INFO [http-nio-8080-exec-7] INFO [http-nio-8080-exec-7] INFO [http-nio-8080-exec-9] INFO [http-nio-8080-exec-9] INFO [http-nio-8080-exec-7] INFO [http-nio-8080-exec-7] INFO [http-nio-8080-exec-7] INFO [http-nio-8080-exec-3] INFO [http-nio-8080-exec-7] INFO [http-nio-8080-exec-3] INFO The configurations for tomcat remains same as default. Session timeout is set as 30 min (default). For more info please let me know. Hope will get some help from here. Thanks! regards, Ripu daman
