Re: Tomcat shutdown password complexity
calder wrote: We've never had occasion to use the password, because we disable shutdown (the better option). Never did understand this Tomcat oddity. What other application is configured by default to open a tcp socket just to receive a shutdown command? Then there the default password, both of which, IMO, warrant a CVE. Would be far better i.e. more standards-based and secure, if the socket were an option and the default stop method was, like everything else, to use rc/init/service/systemctl/whatever. OTOH, a quick look at the startup, shutdown, catalina, ... scripts, much less their lack of reliability, makes a little clearer why some devops might want to avoid the shipped daemon control scripts. Roger Marquis - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: APR connector questions
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Daniel,
On 5/9/20 12:34, daniel@dell.com wrote:
> We want to use APR to call openssl also do with native to support
FIPS mode in tomcat.
>
> Software info Tomcat/9.0.34 libtcnative-1-0-1.2.23-15.30.x86_64
Where did you get that? Is it tcnative-1.2.23?
What about your APR version?
> configuration as below:
>
> connectionTimeout="6" maxKeepAliveRequests="150"
> SSLCertificateFile="*" SSLCertificateChainFile=""
> SSLCertificateKeyFile="*" compression="on"
>
compressibleMimeType="text/html,text/xml,text/css,text/javascript,applic
ation/javascript"
> port="${bio.https.port}"
> protocol="org.apache.coyote.http11.Http11AprProtocol"
> scheme="https" secure="true" sslProtocol="TLS"
> sslEnabledProtocols="TLSv1.2" URIEncoding="UTF-8"/>
>
>
> When enable debug info in tomcat will see
>
> 09-May-2020 00:51:35.358 FINE [https-openssl-apr-8443-exec-1]
org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper.doClose
Calling
[org.apache.tomcat.util.net.AprEndpoint@4275c20c].closeSocket([org.apach
e.tomcat.util.net.AprEndpoint$AprSocketWrapper@1efb5c3e:139622944367568]
)
> 09-May-2020 00:51:35.367 FINE [https-openssl-apr-8443-Poller]
org.apache.tomcat.util.net.AprEndpoint$Poller.removeFromPoller
Attempting to remove [139,622,944,367,568] from poller
Woah, that looks super weird.
> 09-May-2020 00:51:35.367 FINER [https-openssl-apr-8443-Poller]
org.apache.tomcat.util.net.AprEndpoint.destroySocketInternal
Destroying socket [139,622,944,367,568]
> java.lang.Exception at
org.apache.tomcat.util.net.AprEndpoint.destroySocketInternal(AprEndpoint
.java:758)
> at
org.apache.tomcat.util.net.AprEndpoint.access$200(AprEndpoint.java:81)
> at
org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1338)
> at java.base/java.lang.Thread.run(Thread.java:834)
Anything before that in the logs? I mean ... anything relevant?
- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl63KogACgkQHPApP6U8
pFjbuQ/+OZN86zV/0GY0KWYuEABogZi7JIhAOf+OWzyM12f55ka0fdVkO3PeAeQy
yoxqdfvWIswgghNFkXivr3gzsjou1rHbSoCHEDJdBHxU/iuz5BxWjrJcibX2Bejh
OKFNwvp3mq9lxSM/hActSATxXRJJ9p8CPph5qsjaFPmVB4Xnl6rn3295/rJ00kJ/
og4M7RfD4zaftcV9qWyqHTxgJ1xxYIr32Qmh6dVoL6nn/K0uiQeXIBseJAV8wZvs
7QBr3cAq+MSkGacP+64zAVYld/w0wpQt3a2+FiQbT1dNzUxoYG6B0SbJtHbda9on
rT00MSTY1aNxZvc/h+zjuOdd9YmeM7iyeOfHkHymFtmZY/TnJv0Y9mQoA+8u5W9o
/cFR69s1nRKqwLyEQss6MqeaDMbXPiycrz2xPJqqCr7SyzfmNetpOnBvyDuGRZuS
U+rSDopVvFkyugm9HvoJkhCMqBTWaUZ53kwYuQysgObfJZTITuht+iOcidjRQZPC
5sM2dhUpgx6g0ClX6rVUlBzBEJneG/c0pRhA4nNcd90ymQkirti+du9DdLLVy6Sq
UYOYOq2HJk0qQhOEDXjDI/Kx5S4KbaWLxIQH4131kS3QSA7rw123DRRH58fmFQrM
7m1felpLgCscMJqRkMOAr4o54yw7kUteq98hoMCL8s+E06KDiGs=
=dWlU
-END PGP SIGNATURE-
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat shutdown password complexity
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Calder, On 5/9/20 00:36, calder wrote: > On Fri, May 8, 2020 at 9:07 PM calder > wrote: >> >> On Fri, May 8, 2020, 19:20 Robert Hicks >> wrote: >>> >>> I am trying to find what the password complexity can be. I've >>> looked at several hardening guides and they are all >>> "WordsLikeThis". Does the shutdown password take symbols and >>> numbers or at least hyphenated words? >> >> >> We've never had occasion to use the password, because we disable >> shutdown (the better option). >> >> However, my best guess one could use anything. One could check >> the source code, or better yet, set up a Dev instance and give it >> a quick test - a 15 minute exercise at most. > > Gave it a test. > > In server.xml, we have > > > and then fire it up > > user@stimpy:~/bin/apache-tomcat/bin> ./catalina.sh start > log.log > 2>&1 > > user@stimpy:~/bin/apache-tomcat/bin> ps aux | grep java user 7223 > 531 1.2 21006280 812812 pts/2 Sl 23:22 0:13 /home/ [ ... ] > > user@stimpy:~/bin/apache-tomcat/bin> ./shutdown.sh stop > > user@stimpy:~/bin/apache-tomcat/bin> ps aux | grep "bin/java" [ no > response ] > > If we start up TC and change server.xml entry to (removed one char > at end) TC won't shut > down. > > Keep in mind - some characters won't work like & or ( or ) - at > least on Unix-style OSes as the shell may want to interpret them. What makes you say that? What does the shell have to do with anything? - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl63KfYACgkQHPApP6U8 pFh5yRAAmIInP54+INuiba2Hbjb/AxmqqNMrmP6noARMyPCuOL6ptjumqvebT1J8 tw7oIPJPT3qEFzg2TvXZ/QJ/sQ6or9/Q1PYZ8eZnEtv4Cw5LMSmgLV/69MAMhtfA o6X0V7ZdKwpnLhfIvV8we/kogmfD2h5gqHmqtL165pbBO5FzqywNUJoYIaOaiNtk 9ExWHWZ/+pRxwfS7OkrVLYn9UlIKebFJX1fAqjAMGFnAcI45L5ky6oRjpY359UfJ tQDXbmsu034TGnLdrnhiSGASWHGEPsTmaH2m2o24WW0Sf75ymEsWVkV9RGOYsyAG lBtX7Bj4fa0Ldr/S4ejXEBy7p+e+t+5BNw8yUZKSyE9zPwL77Yp23hL2w83hUQbq beNNIia7HaDpO3x9ZaRT53UALNVTnKdJNmTfIHHPm5m8WAeaaJz7vKHcRdWtkZSg 4GZ1TW5VXnwL27jxSnYlDTBM6o/xUAuVc8ZmpYt2U7fFKnQVE57mVn8BG+jFLPI4 19F6jjIL7bzqIhx4h26af5xeYeqXWLeWRzZWA+nS9GpoPkYFTfmGByGS54bKU0rE lMd/3nRKcjt+PMVM7wnu8b/S+hrSTwG1nE3ens9XPwpJCl0HsZzX5HR51SJegOXF O2xOeuy9as1+jAGtquiQpvOZePDbrGUjJaZebZ4fQE0+acJ1bo4= =JGZQ -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat request hooks
Hi Mark, Thanks for the detailed reply. I think I get it now. The problem I was trying to solve was to dynamically control the concurrency/in-flight requests (requests being processed by worker threads + requests in queue) based on the response times of requests (time spent in queue + time spent in worker threads). On Sat, May 9, 2020 at 8:12 PM Mark Thomas wrote: > On 09/05/2020 14:08, Vikas Kumar wrote: > > I did not know about the images being dropped. > > > > Here are the steps I depicted in the image: > > > > Step 0: Request accepted by OS > > Step 1: Request sent to Tomcat. Tomcat accepts the request if it's > internal > > queue is not full, else rejects the request > > Step 2: Tomcat adds the request to the queue if all worker threads are > > busy, else sends the request to one of the free worker threads > > Step 3: Request is picked by the worker thread for processing > > The above sequence isn't quite right. > > Step 0 is the accept queue. See acceptCount in the docs. Note that not > every OS honours this setting and may use a smaller or larger value. It > the accept queue is full, the connection gets dropped. > > Step 1 is the Acceptor thread. This runs in a fairly tight loop. If the > current connection count > maxConnections, the loop pauses. The Acceptor > thread takes the next connection from the accept queue, accepts it > (creates the connection) and then creates a task for the executor to > process that connection. It then loops. If the connection isn't accepted > in time, it will time out. The queue for the executor is essentially > unlimited - although in practice it should never exceed maxConnections > > Step 2 Is the executor. If there are tasks in the queue and spare worker > threads available, the task (== connection) is allocated a worker thread > for processing. > > Step 3 is that the executor/worker thread processes the connection. When > the request is completed, the connection is either kept open (e.g. HTTP > keep alive) or closed. > > > I would like to add a hook between step(1) and step (2). As I mentioned > > earlier, I want to dynamically control the Tomcat internal queue size > based > > on some application logic. Is there a way to do that? > > Do you want to control the acceptCount, maxConnections or something else? > > What problem are you trying to solve? > > The overhead of the accept process and the allocation of requests to > workers is fairly light weight. I wonder if the simplest solution would > be to allow every request through to the application and then decide > whether to process it or reject it. You could provide an HTTP error > response at that point that might be more informative for your users. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
RE: APR connector questions
We want to use APR to call openssl also do with native to support FIPS mode in tomcat. Software info Tomcat/9.0.34 libtcnative-1-0-1.2.23-15.30.x86_64 configuration as below: When enable debug info in tomcat will see 09-May-2020 00:51:35.358 FINE [https-openssl-apr-8443-exec-1] org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper.doClose Calling [org.apache.tomcat.util.net.AprEndpoint@4275c20c].closeSocket([org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper@1efb5c3e:139622944367568]) 09-May-2020 00:51:35.367 FINE [https-openssl-apr-8443-Poller] org.apache.tomcat.util.net.AprEndpoint$Poller.removeFromPoller Attempting to remove [139,622,944,367,568] from poller 09-May-2020 00:51:35.367 FINER [https-openssl-apr-8443-Poller] org.apache.tomcat.util.net.AprEndpoint.destroySocketInternal Destroying socket [139,622,944,367,568] java.lang.Exception at org.apache.tomcat.util.net.AprEndpoint.destroySocketInternal(AprEndpoint.java:758) at org.apache.tomcat.util.net.AprEndpoint.access$200(AprEndpoint.java:81) at org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1338) at java.base/java.lang.Thread.run(Thread.java:834) BRs Dan -Original Message- From: Christopher Schultz Sent: Friday, May 8, 2020 10:37 PM To: users@tomcat.apache.org Subject: Re: APR connector questions -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Daniel, On 5/8/20 04:25, daniel@dell.com wrote: > We are changing from Nio connector to APR connector to enable FIPS > mode in tomcat. But we hit tomcat hang issue, ssl handshake no > response when run long time. So many close_wait in netstat output. > Do you have any advises about that issue? Can you please post your configuration? Remember to remove any secrets that may be in there. You may be interested to know that FIPS is available through Java, though not through Sun's JSSE provider. https://stackoverflow.com/questions/5046482/which-jce-providers-are-fips - -140-2-compliant You may also be interested in the fact that FIPS mode doesn't really offer any additional security. In certain cases, it may reduce your security because of the various required-supported algorithms which, honestly, should never be used in production. - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl61bnAACgkQHPApP6U8 pFjf2Q/+K/kHIF36pSJ3gzU1gkrRnmDOqLtNX4rAzJVguZrOqSDjVNyFjYlYPcDD A9szjfgdwd8PlTdgXJISpvdSqdvjGSadKbNswcN731VDptMlUz979R54+kRHeoWU lYdwZuNp/ACj+UXJnSDcxK0Q15UewlRLuTrtpFfoCkteS1uAXAH1OMStsZYFXrSt Jc3XmrmidTfAl8P24W8xNFxCTDPhkcnO7nJaNPKlGwdtjtxVfOaxyK9UtoKJW+te lANt3Fi8r5QlLbZIofK9A0BTyHsk17SmUseeETDPCUcqlEZ1z8KWN6NVlLl0O4Rk P/i3JUrsD8ZuCMghj1Jw6s4B4aWolLoSvxFYGLmNitqGNPGQnuUid5RV6LWLW7nH kMFDE6yGXHagZ/34GIWcPVJOmcobOdFGtGXb4SWRsf9xOU8U5g2ljpSIYA0xT4J+ lCWZLxkcxW0YdppfPWU7t7uKO8GPnCjBmBUgx7fSHRvNefrgof6CRSAjyKlMsU1w WSW8ZPblXSBToHy98JoT27wTrYUkhfDGzCDopkMxGH4QZZtvIVH+MNsBpWUWMhMc h/yo2ubKWwsrmPBhkd+Jjkon3FGsuBRpUdNQJx0+5G5CKGuDNFIIZYV5MDK0ovCu wmBN/6ZSwUj7ZqpOFekGHhM4DUee8R0kXmScDXd1nogkoIGIO20= =JFpT -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat request hooks
On 09/05/2020 14:08, Vikas Kumar wrote: > I did not know about the images being dropped. > > Here are the steps I depicted in the image: > > Step 0: Request accepted by OS > Step 1: Request sent to Tomcat. Tomcat accepts the request if it's internal > queue is not full, else rejects the request > Step 2: Tomcat adds the request to the queue if all worker threads are > busy, else sends the request to one of the free worker threads > Step 3: Request is picked by the worker thread for processing The above sequence isn't quite right. Step 0 is the accept queue. See acceptCount in the docs. Note that not every OS honours this setting and may use a smaller or larger value. It the accept queue is full, the connection gets dropped. Step 1 is the Acceptor thread. This runs in a fairly tight loop. If the current connection count > maxConnections, the loop pauses. The Acceptor thread takes the next connection from the accept queue, accepts it (creates the connection) and then creates a task for the executor to process that connection. It then loops. If the connection isn't accepted in time, it will time out. The queue for the executor is essentially unlimited - although in practice it should never exceed maxConnections Step 2 Is the executor. If there are tasks in the queue and spare worker threads available, the task (== connection) is allocated a worker thread for processing. Step 3 is that the executor/worker thread processes the connection. When the request is completed, the connection is either kept open (e.g. HTTP keep alive) or closed. > I would like to add a hook between step(1) and step (2). As I mentioned > earlier, I want to dynamically control the Tomcat internal queue size based > on some application logic. Is there a way to do that? Do you want to control the acceptCount, maxConnections or something else? What problem are you trying to solve? The overhead of the accept process and the allocation of requests to workers is fairly light weight. I wonder if the simplest solution would be to allow every request through to the application and then decide whether to process it or reject it. You could provide an HTTP error response at that point that might be more informative for your users. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat request hooks
On Sat, 9 May 2020 at 9:09 PM, Vikas Kumar wrote: > I did not know about the images being dropped. > > Here are the steps I depicted in the image: > > Step 0: Request accepted by OS > Step 1: Request sent to Tomcat. Tomcat accepts the request if it's internal > queue is not full, else rejects the request > Step 2: Tomcat adds the request to the queue if all worker threads are > busy, else sends the request to one of the free worker threads > Step 3: Request is picked by the worker thread for processing > > I would like to add a hook between step(1) and step (2). As I mentioned > earlier, I want to dynamically control the Tomcat internal queue size based > on some application logic. Is there a way to do that? > The queue is managed here: https://github.com/apache/tomcat/blob/14406c0b49c29fd05dd8f707b62ece38429e16f8/java/org/apache/tomcat/util/net/AbstractEndpoint.java#L429 And there is an mbean here for the NIO connector: https://github.com/apache/tomcat/blob/7d66c652a3e4d64b9711712fa1409ba1261effd8/java/org/apache/tomcat/util/net/mbeans-descriptors.xml#L91 I don’t know if that makes the attribute query able as well as writable, though. You may want to experiment and check. What you are looking at writing, though, will be code that lives outside your warfile and as an extension to Tomcat ( is you want to enhance the AbstractEndpoint above), or at least JMX privileges (in case changing the limit is permitted by that mbean config) Alternatively, you could consider setting maxConnections and or acceptCount to higher numbers. See : https://tomcat.apache.org/tomcat-8.5-doc/config/http.html Ram > > > > > > > > > > > > > > > On Sat, May 9, 2020 at 6:16 PM calder wrote: > > > On Sat, May 9, 2020, 07:16 Vikas Kumar wrote: > > > > > As per my understanding (using a Spring Boot app with Tomcat server), > we > > > define: > > > > > >- Max no. of worker threads (maxThreads, default 200) > > >- Tomcat queue size (maxConnections, default 8192 for APR, 1 for > > >NIO). When all worker threads are busy, requests are placed into the > > queue. > > >As worker threads free up, queued requests are sent to them in FIFO > > order > > > > > > > > I'll assume the "steps" are shown in the "broken" image (inline/attached > > images are normally dropped) - you'll need the convert whatever text is > > contained into actual text and post back. > > > > > > [image: Untitled Diagram.png] > > > > > > I can add a hook at step (3) using a servlet filter or > > > > > [ snip ] > > >
Re: Tomcat request hooks
I did not know about the images being dropped. Here are the steps I depicted in the image: Step 0: Request accepted by OS Step 1: Request sent to Tomcat. Tomcat accepts the request if it's internal queue is not full, else rejects the request Step 2: Tomcat adds the request to the queue if all worker threads are busy, else sends the request to one of the free worker threads Step 3: Request is picked by the worker thread for processing I would like to add a hook between step(1) and step (2). As I mentioned earlier, I want to dynamically control the Tomcat internal queue size based on some application logic. Is there a way to do that? On Sat, May 9, 2020 at 6:16 PM calder wrote: > On Sat, May 9, 2020, 07:16 Vikas Kumar wrote: > > > As per my understanding (using a Spring Boot app with Tomcat server), we > > define: > > > >- Max no. of worker threads (maxThreads, default 200) > >- Tomcat queue size (maxConnections, default 8192 for APR, 1 for > >NIO). When all worker threads are busy, requests are placed into the > queue. > >As worker threads free up, queued requests are sent to them in FIFO > order > > > > > I'll assume the "steps" are shown in the "broken" image (inline/attached > images are normally dropped) - you'll need the convert whatever text is > contained into actual text and post back. > > > [image: Untitled Diagram.png] > > > > I can add a hook at step (3) using a servlet filter or > > > [ snip ] >
Re: Tomcat request hooks
On Sat, May 9, 2020, 07:16 Vikas Kumar wrote: > As per my understanding (using a Spring Boot app with Tomcat server), we > define: > >- Max no. of worker threads (maxThreads, default 200) >- Tomcat queue size (maxConnections, default 8192 for APR, 1 for >NIO). When all worker threads are busy, requests are placed into the queue. >As worker threads free up, queued requests are sent to them in FIFO order > > I'll assume the "steps" are shown in the "broken" image (inline/attached images are normally dropped) - you'll need the convert whatever text is contained into actual text and post back. [image: Untitled Diagram.png] > > I can add a hook at step (3) using a servlet filter or > [ snip ]
Tomcat request hooks
As per my understanding (using a Spring Boot app with Tomcat server), we define: - Max no. of worker threads (maxThreads, default 200) - Tomcat queue size (maxConnections, default 8192 for APR, 1 for NIO). When all worker threads are busy, requests are placed into the queue. As worker threads free up, queued requests are sent to them in FIFO order [image: Untitled Diagram.png] I can add a hook at step (3) using a servlet filter or request interceptor for pre and post-processing of the request. My question is: can I add a similar hook at step (2)? Basically I want to dynamically control the Tomcat queue size based on some application logic.
