Thanks Martin.
But with the old version i.e. 8.5.24 it is working smoothly. So, what could
be the problem? Or some specific property/configuration changes that need
to be made around this?
On Thu, Jul 23, 2020 at 6:00 PM Martin Grigorov <mgrigo...@apache.org>
wrote:
> On Thu, Jul 23, 2020 at 3:10 PM Kushagra Bindal <bindal.kusha...@gmail.com
> >
> wrote:
>
> > Hi Martin,
> >
> > Due to our environment I was not able to use pastebin service. I have
> > taken different Thread dump during shutdown and attaching the same with
> > this email.
> >
> > Please review the same and let me know, what is the probable root cause
> of
> > the problem and what could be the fix of the same.
> >
>
> You have many AMQP (RabbitMQ) listener threads which are not daemons.
> It seems your application does not notify Spring Framework that it needs to
> destroy its application context or the beans with
> type
> org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer
> are not notified to stop for some other
> reason.
>
>
> >
> > On Thu, Jul 23, 2020 at 3:22 PM Martin Grigorov <mgrigo...@apache.org>
> > wrote:
> >
> >> Hi,
> >>
> >> On Thu, Jul 23, 2020 at 6:35 AM Kushagra Bindal <
> >> bindal.kusha...@gmail.com>
> >> wrote:
> >>
> >> > Hi Martin,
> >> >
> >> > These are the only two behaviors right now which I am getting on a
> >> > regular basis.
> >> > 1. During startup of the application and then shutdown
> >> > 2. Running application and then shutdown.
> >> >
> >> > Please let me know if anything specific is further needed from my end
> >> which
> >> > I can provide to have a better clarity.
> >> >
> >> > I have shared the server.xml and command which we are using in
> stopping
> >> the
> >> > tomcat.
> >> >
> >> >
> >> > On Thu, Jul 23, 2020 at 2:49 AM Martin Grigorov <mgrigo...@apache.org
> >
> >> > wrote:
> >> >
> >> > > On Wed, Jul 22, 2020, 15:55 Kushagra Bindal <
> >> bindal.kusha...@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Hi Christopher,
> >> > > >
> >> > > > Did you get a chance to look into this?
> >> > > >
> >> > > > Please help us in resolving this issue.
> >> > > >
> >> > > > On Sat, Jul 18, 2020 at 11:26 AM Kushagra Bindal <
> >> > > > bindal.kusha...@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Chris,
> >> > > > >
> >> > > > > Additionally when trying to stop running application, we are
> >> getting
> >> > > > below
> >> > > > > error.
> >> > > > >
> >> > > > > Sat Jul 18 05:49:40 UTC 2020
> >> > > > > **************************
> >> > > > > * Stopping the Web Server
> >> > > > > **************************
> >> > > > > Sat Jul 18 05:49:40 UTC 2020
> >> > > > > ./wfc: line 28: /usr/local/nginx/nginx: No such file or
> directory
> >> > > > > ./wfc: line 233: /usr/local/nginx/nginx: No such file or
> directory
> >> > > > >
> >> > > > > Sat Jul 18 05:49:40 UTC 2020
> >> > > > > * Nginx has been stopped
> >> > > > > **************************
> >> > > > > * Shutdown the wfc Server gracefully
> >> > > > > **************************
> >> > > > > #
> >> > *********************************************************************
> >> > > > > # Tomcat shutdown with wait time: 30
> >> > > > > #
> >> > *********************************************************************
> >> > > > > Using CATALINA_BASE: /usr/local/xyz/tomcat
> >> > > > > Using CATALINA_HOME: /usr/local/xyz/tomcat
> >> > > > > Using CATALINA_TMPDIR: /usr/local/xyz/tomcat/temp
> >> > > > > Using JRE_HOME: /usr/local/xyz/jdk
> >> > > > > Using CLASSPATH:
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> /usr/local/xyz/tomcat/bin/bootstrap.jar:/usr/local/xyz/tomcat/bin/tomcat-juli.jar
> >> > > > > Using CATALINA_PID: /usr/local/xyz/tomcat/tomcat.pid
> >> > > > > Tomcat did not stop in time.
> >> > > > > To aid diagnostics a thread dump has been written to standard
> out.
> >> > > >
> >> > >
> >> > > Anything interesting in this thread dump?
> >> >
> >>
> >> Tomcat does not stop properly in time and a thread dump is written to
> the
> >> stdout.
> >> Please share the thread dump. Use some pastebin service!
> >>
> >>
> >> > >
> >> > >
> >> > > > Killing Tomcat with the PID: 4280
> >> > > > > The Tomcat process has been killed.
> >> > > > > #
> >> > *********************************************************************
> >> > > > > # Tomcat shutdown result: 0
> >> > > > > #
> >> > *********************************************************************
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > Whereas below error that I mentioned earlier is coming when
> >> > application
> >> > > > is
> >> > > > > in INPROGRESS and not yet come into running state and when we
> stop
> >> > the
> >> > > > same
> >> > > > > it is throwing below error.
> >> > > > >
> >> > > > > On Sat, Jul 18, 2020 at 10:17 AM Kushagra Bindal <
> >> > > > > bindal.kusha...@gmail.com> wrote:
> >> > > > >
> >> > > > >> Hi Chris,
> >> > > > >>
> >> > > > >> To stop tomcat we are using the below command.
> >> > > > >>
> >> > > > >> bin/shutdown.sh -$sleeptime -force
> >> > > > >>
> >> > > > >> Where in our case sleeptime is set to 30.
> >> > > > >>
> >> > > > >> Please find the attached server.xml which we are using in our
> >> > > > application.
> >> > > > >>
> >> > > > >> Also I have copy-paste the complete content below in this email
> >> as
> >> > > well.
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> ----------------------------------------------------------------------------------------
> >> > > > >>
> >> > > > >> <?xml version='1.0' encoding='utf-8'?>
> >> > > > >> <Server port="8005" shutdown="SHUTDOWN">
> >> > > > >> <Listener
> >> > > > className="org.apache.catalina.startup.VersionLoggerListener"
> >> > > > >> />
> >> > > > >> <Listener
> >> > className="org.apache.catalina.core.AprLifecycleListener"
> >> > > > >> SSLEngine="on" />
> >> > > > >> <Listener
> >> > > > >>
> >> className="org.apache.catalina.core.JreMemoryLeakPreventionListener"
> >> > > />
> >> > > > >> <Listener
> >> > > > >>
> >> > >
> >> className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener"
> >> > > > />
> >> > > > >> <Listener
> >> > > > >>
> >> > className="org.apache.catalina.core.ThreadLocalLeakPreventionListener"
> >> > > > />
> >> > > > >> <GlobalNamingResources>
> >> > > > >> <Resource name="UserDatabase" auth="Container"
> >> > > > >> type="org.apache.catalina.UserDatabase"
> >> > > > >> description="User database that can be updated
> and
> >> > > saved"
> >> > > > >>
> >> > > > >> factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
> >> > > > >> pathname="conf/tomcat-users.xml" />
> >> > > > >> <Resource name="jdbc/edbDataSource" auth="Container"
> >> > > > >> type="javax.sql.DataSource"
> >> > > > >> username="db_username_stub"
> >> > > > >> password="db_password_stub"
> >> > > > >>
> >> > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
> >> > > > >> url="READ_WRITE_URL_STUB"
> >> > > > >> driverClassName="com.edb.Driver"
> >> > > > >> initialSize="5"
> >> > > > >> maxWait="30000"
> >> > > > >> maxActive="50"
> >> > > > >> maxIdle="20"
> >> > > > >> minIdle="5"
> >> > > > >> maxAge="7200000"
> >> > > > >> validationQuery="SELECT 1; COMMIT;"
> >> > > > >> initSQL="ALTER SESSION SET statement_timeout=3600000; ALTER
> >> SESSION
> >> > > SET
> >> > > > >> idle_in_transaction_session_timeout=3660000; COMMIT;"
> >> > > > >> poolPreparedStatements="true"
> >> > > > >> testWhileIdle="false"
> >> > > > >> testOnBorrow="true"
> >> > > > >> testOnReturn="false"
> >> > > > >> validationInterval="120000"
> >> > > > >> timeBetweenEvictionRunsMillis="15000"
> >> > > > >> removeAbandonedTimeout="300"
> >> > > > >> removeAbandoned="false"
> >> > > > >> logAbandoned="false"
> >> > > > >> minEvictableIdleTimeMillis="120000"
> >> > > > >> jmxEnabled="true"
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;
> >> > > > >>
> >> > > > >> org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer;
> >> > > > >>
> >> > > > >>
> >> org.apache.tomcat.jdbc.pool.interceptor.StatementCache(max=4000)" />
> >> > > > >> <Resource name="jdbc/edbDataSourceReadOnly"
> auth="Container"
> >> > > > >> type="javax.sql.DataSource"
> >> > > > >> username="db_username_stub"
> >> > > > >> password="db_password_stub"
> >> > > > >>
> >> > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
> >> > > > >> url="READ_ONLY_URL_STUB"
> >> > > > >> driverClassName="com.edb.Driver"
> >> > > > >> initialSize="5"
> >> > > > >> maxWait="30000"
> >> > > > >> maxActive="50"
> >> > > > >> maxIdle="20"
> >> > > > >> minIdle="5"
> >> > > > >> defaultReadOnly="true"
> >> > > > >> validationQuery="SELECT 1; COMMIT;"
> >> > > > >> initSQL="ALTER SESSION SET
> >> statement_timeout=3600000;
> >> > > > ALTER
> >> > > > >> SESSION SET idle_in_transaction_session_timeout=3660000;
> COMMIT;"
> >> > > > >> poolPreparedStatements="true"
> >> > > > >> testWhileIdle="false"
> >> > > > >> testOnBorrow="true"
> >> > > > >> testOnReturn="false"
> >> > > > >> validationInterval="120000"
> >> > > > >> timeBetweenEvictionRunsMillis="15000"
> >> > > > >> removeAbandonedTimeout="300"
> >> > > > >> removeAbandoned="false"
> >> > > > >> logAbandoned="false"
> >> > > > >> minEvictableIdleTimeMillis="120000"
> >> > > > >> jmxEnabled="true"
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;
> >> > > > >>
> >> > > > >> org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer;
> >> > > > >>
> >> > > > >>
> >> org.apache.tomcat.jdbc.pool.interceptor.StatementCache(max=4000)" />
> >> > > > >> </GlobalNamingResources>
> >> > > > >> <Service name="Catalina">
> >> > > > >> <Connector port="8080" protocol="HTTP/1.1"
> >> > > > >> server=""
> >> > > > >> connectionTimeout="20000"
> >> > > > >> redirectPort="8443" maxHttpHeaderSize="16384"/>
> >> > > > >> <Connector port="8085" protocol="HTTP/1.1"
> >> > > > >> server=""
> >> > > > >> connectionTimeout="20000"
> >> > > > >> redirectPort="8443" maxHttpHeaderSize="16384"/>
> >> > > > >>
> >> > > > >> <Engine name="Catalina" defaultHost="localhost">
> >> > > > >> <Realm
> className="org.apache.catalina.realm.LockOutRealm">
> >> > > > >> <Realm
> >> > className="org.apache.catalina.realm.UserDatabaseRealm"
> >> > > > >> resourceName="UserDatabase"/>
> >> > > > >> </Realm>
> >> > > > >> <Host name="localhost" appBase="webapps"
> >> > > > >> unpackWARs="true" autoDeploy="true">
> >> > > > >> <Valve
> >> className="org.apache.catalina.valves.AccessLogValve"
> >> > > > >> directory="${log.dir}/tomcat"
> >> > > > >> prefix="localhost_access_log" suffix=".txt"
> >> > > > >> pattern="%h %l %u %t "%r" %s %b" />
> >> > > > >> </Host>
> >> > > > >> </Engine>
> >> > > > >> </Service>
> >> > > > >> </Server>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> -----------------------------------------------------------------------------------------
> >> > > > >>
> >> > > > >> Please suggest what needs to be done to resolve this issue.
> >> > > > >>
> >> > > > >>
> >> > > > >> On Sat, Jul 18, 2020 at 12:47 AM Christopher Schultz <
> >> > > > >> ch...@christopherschultz.net> wrote:
> >> > > > >>
> >> > > > >>> -----BEGIN PGP SIGNED MESSAGE-----
> >> > > > >>> Hash: SHA256
> >> > > > >>>
> >> > > > >>> Kushagra,
> >> > > > >>>
> >> > > > >>> On 7/17/20 11:47, Kushagra Bindal wrote:
> >> > > > >>> > Similar issues we are also facing in our environment. What
> >> could
> >> > be
> >> > > > >>> > the problem?
> >> > > > >>> >
> >> > > > >>> > On Fri, Jul 17, 2020 at 6:04 PM om tiwari <
> >> omt121...@gmail.com>
> >> > > > >>> > wrote:
> >> > > > >>> >
> >> > > > >>> >> Hi,
> >> > > > >>> >>
> >> > > > >>> >> We have upgraded our tomcat version from 8.5.24 to 8.5.53
> in
> >> > > > >>> >> application. After upgrading we are facing issue while
> >> stopping
> >> > > > >>> >> tomcat. Below is the stack-trace :
> >> > > > >>> >>
> >> > > > >>> >> Using CATALINA_BASE: /usr/local/kronos/tomcat Using
> >> > > > >>> >> CATALINA_HOME: /usr/local/kronos/tomcat Using
> >> CATALINA_TMPDIR:
> >> > > > >>> >> /usr/local/kronos/tomcat/temp Using JRE_HOME:
> >> > > > >>> >> /usr/local/kronos/jdk Using CLASSPATH:
> >> > > > >>> >>
> >> > > > >>> >>
> >> > > >
> >> /usr/local/kronos/tomcat/bin/bootstrap.jar:/usr/local/kronos/tomcat/b
> >> > > > >>> in/tomcat-juli.jar
> >> > > > >>> >>
> >> > > > >>> >>
> >> > > > >>> Using CATALINA_PID: /usr/local/kronos/tomcat/tomcat.pid
> >> > > > >>> >> Jul 17, 2020 11:44:44 AM
> org.apache.catalina.startup.Catalina
> >> > > > >>> >> stopServer SEVERE: Could not contact [localhost:8005].
> Tomcat
> >> > may
> >> > > > >>> >> not be running. Jul 17, 2020 11:44:44 AM
> >> > > > >>> >> org.apache.catalina.startup.Catalina stopServer SEVERE:
> >> > > > >>> >> Catalina.stop: java.net.ConnectException: Connection
> refused
> >> > > > >>> >> (Connection refused) at
> >> > > > >>> >> java.net.PlainSocketImpl.socketConnect(Native Method) at
> >> > > > >>> >> java.net
> >> > > > >>> >>
> >> > > .AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> >> > > > >>> >>
> >> > > > >>> >>
> >> > > > >>> at java.net
> >> > > > >>> >>
> >> > > >
> >> .AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.jav
> >> > > > >>> a:206)
> >> > > > >>> >>
> >> > > > >>> >>
> >> > > > >>> at java.net
> >> > > > >>> >>
> >> > .AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> >> > > > >>> >>
> >> > > > >>> >>
> >> > > > >>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> >> > > > >>> >> at java.net.Socket.connect(Socket.java:589) at
> >> > > > >>> >> java.net.Socket.connect(Socket.java:538) at
> >> > > > >>> >> java.net.Socket.<init>(Socket.java:434) at
> >> > > > >>> >> java.net.Socket.<init>(Socket.java:211) at
> >> > > > >>> >>
> >> > org.apache.catalina.startup.Catalina.stopServer(Catalina.java:504)
> >> > > > >>> >>
> >> > > > >>> >>
> >> > > > >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> > > > >>> >> at
> >> > > > >>> >>
> >> > > >
> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
> >> > > > >>> java:62)
> >> > > > >>> >>
> >> > > > >>> >>
> >> > > > >>> at
> >> > > > >>> >>
> >> > > >
> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> >> > > > >>> sorImpl.java:43)
> >> > > > >>> >>
> >> > > > >>> >>
> >> > > > >>> at java.lang.reflect.Method.invoke(Method.java:498)
> >> > > > >>> >> at
> >> > > > >>> >>
> >> > > org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:389)
> >> > > > >>> >>
> >> > > > >>> >>
> >> > > > >>> at
> >> org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:479)
> >> > > > >>> >>
> >> > > > >>> >> Tomcat did not stop in time. To aid diagnostics a thread
> dump
> >> > has
> >> > > > >>> >> been written to standard out. Killing Tomcat with the PID:
> >> 21210
> >> > > > >>> >> The Tomcat process has been killed.
> >> > > > >>> >>
> >> > > > >>> >>
> >> > > >
> >> ---------------------------------------------------------------------
> >> > > > >>> - ---------------------------------
> >> > > > >>> >>
> >> > > > >>> >>
> >> > > > >>> >>
> >> > > > >>> At last the tomcat is stopped but we are not able to
> understand
> >> > this
> >> > > > >>> >> connection refused stack trace in this stop process.
> >> > > > >>> >>
> >> > > > >>> >> It also takes time in stopping the tomcat server now since
> >> > > > >>> >> upgrade `10-15 sec.
> >> > > > >>> >>
> >> > > > >>> >> Can anyone help us in resolving this issue?
> >> > > > >>>
> >> > > > >>> Can you post your <Server> element from conf/server.xml?
> >> > > > >>>
> >> > > > >>> How do you stop Tomcat -- looks like using "catalina.sh stop"
> or
> >> > > > similar
> >> > > > >>> ?
> >> > > > >>>
> >> > > > >>> - -chris
> >> > > > >>> -----BEGIN PGP SIGNATURE-----
> >> > > > >>> Comment: Using GnuPG with Thunderbird -
> >> https://www.enigmail.net/
> >> > > > >>>
> >> > > > >>>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl8R9N4ACgkQHPApP6U8
> >> > > > >>>
> pFga8RAAws1xIkDJJQptsdTu/2T/jssbnnDVewrI4LXguNoavHBxncGzw8r3Jczw
> >> > > > >>>
> Qkx7rJcpoP9KcBmvDjipLDimVMS/uqErAgzb1kS52qUD44gdL2HdKfTyvVmUnOGU
> >> > > > >>>
> xqlVD6JrJjLhATxhHhomDp5omcy8dn+j5XHJD0bGbISjKMsH5SPNUHNI0x2NIWNX
> >> > > > >>>
> u3qcQxNlKiCN5iZpniAPhjVOPA+EFW91W83H37LZkrA7YFV9q4Z/FuHbHit0DL+J
> >> > > > >>>
> sJN7oW9afPeVco5Q80IKiT2EDQEhcLH2zUNRwYqGfktewsgOM5cWN8NSXmqqsZDz
> >> > > > >>>
> B2Zq7OqhAGx3b/OV9OpVSO0v+Pv6G2GMmhdv35eFq/JTkJBwMhd8H9PEc8yDEbTF
> >> > > > >>>
> 0VaKmzjWuxs4ALPvD7yQtyidZE0I8H1MQBg4E1ROC1vMNfEkQXT47ChVq4zvJJDv
> >> > > > >>>
> J1h+6ZHCmY6uLkZLltOZQtUnkFUAS0qn6AY80Dm0FWmMcorBUY0XRQSSfY0Xzpmv
> >> > > > >>>
> CafkAgx6TZKiFGTbmXH90JrSrHQBchBvm4GmOKDNp9W3U/ZV8QQoBH4JLsGu/2e4
> >> > > > >>>
> Cgq/TwBmc7VrjW4gYVg/fQ+mkF/4z2K1o9h2vAD/Di50TpARAIrRZbangqwvQxKN
> >> > > > >>> b6IFIT965wW0MuhiaIO1leWc3nqLXzII8EEcg5xsduFx/aqrnKI=
> >> > > > >>> =2Ey1
> >> > > > >>> -----END PGP SIGNATURE-----
> >> > > > >>>
> >> > > > >>>
> >> > ---------------------------------------------------------------------
> >> > > > >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> >> > > > >>> For additional commands, e-mail: users-h...@tomcat.apache.org
> >> > > > >>>
> >> > > > >>>
> >> > > > >>
> >> > > > >> --
> >> > > > >> Regards,
> >> > > > >> Kushagra
> >> > > > >>
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Regards,
> >> > > > > Kushagra
> >> > > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Regards,
> >> > > > Kushagra
> >> > > >
> >> > >
> >> >
> >> >
> >> > --
> >> > Regards,
> >> > Kushagra
> >> >
> >>
> >
> >
> > --
> > Regards,
> > Kushagra
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> > For additional commands, e-mail: users-h...@tomcat.apache.org
>
--
Regards,
Kushagra
