Re: TLSv1.3 Support in Tomcat
https://wiki.openssl.org/index.php/TLS1.3#Ciphersuites TLSv1.3 supports 5 cipher suites and none is in your list. - Daniel Savard Le mar. 29 juin 2021 à 01:44, S Abirami a écrit : > Hi Christopher, > > Below is my Connector element, sslEnabledProtocols =TLSv1.2 ,TLS 1.3 it is > working fine with TLSv1.2. When sslEnabledProtocols=TLSv1.3, Tomcat is > started but, the browser unable to perform handshake with webapp. > > Is there any dependency with Cipher suites? > > protocol="com.ericsson.http.protocol.Http11Nio2ProtocolDecryptProp" > port="" maxThreads="200" scheme="https" secure="true" > SSLEnabled="true" keystoreFile="/opt/cert/keystore" > keystorePass="" clientAuth="false" > maxHttpHeaderSize="8192" server="" xpoweredBy="false" > ciphers="TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, > TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, > TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, > TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, > TLS_DHE_DSS_WITH_AES_256_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, > TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, > TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, > TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, > TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, > TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, > TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, > TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, > TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, > TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, > TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, > TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, > TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, > TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, > TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, > TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, > TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, > TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, > TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, > TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, > TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA" > sslEnabledProtocols=" TLSv1.3"/> > > > > Regards, > Abirami.S > > -Original Message- > From: Christopher Schultz > Sent: Monday, June 28, 2021 7:27 PM > To: users@tomcat.apache.org > Subject: Re: TLSv1.3 Support in Tomcat > > Abirami, > > On 6/28/21 07:16, S Abirami wrote: > > TLSv1.3 support is available in Tomcat. > > > > I tried just updating server.xml[sslEnabledProtocols=TLSv1.3] and > > restarted tomcat. It doesn't work. > > > > [We are using Tomcat 9.0.46 and JDK 8u291] > > > > Please let me know any other configuration also needs to be changed. > > Can you please post your configuration (minus any secrets)? > > When you say "it doesn't work", what exactly do you mean? > > -chris > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
RE: TLSv1.3 Support in Tomcat
Hi Christopher, Below is my Connector element, sslEnabledProtocols =TLSv1.2 ,TLS 1.3 it is working fine with TLSv1.2. When sslEnabledProtocols=TLSv1.3, Tomcat is started but, the browser unable to perform handshake with webapp. Is there any dependency with Cipher suites? Regards, Abirami.S -Original Message- From: Christopher Schultz Sent: Monday, June 28, 2021 7:27 PM To: users@tomcat.apache.org Subject: Re: TLSv1.3 Support in Tomcat Abirami, On 6/28/21 07:16, S Abirami wrote: > TLSv1.3 support is available in Tomcat. > > I tried just updating server.xml[sslEnabledProtocols=TLSv1.3] and > restarted tomcat. It doesn't work. > > [We are using Tomcat 9.0.46 and JDK 8u291] > > Please let me know any other configuration also needs to be changed. Can you please post your configuration (minus any secrets)? When you say "it doesn't work", what exactly do you mean? -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Using log4j for logging
Greetings, I wanted to setup log4j for tomcat logs and google searches seems to indicate that this is possible. Many articles speak about downloading tomcat-juli-adapters.jar from bin/extras directory. I found out that for tomcat version 9, extras directory is last present on version 9.0.14 (https://archive.apache.org/dist/tomcat/tomcat-9/v9.0.14/bin/). Latter versions do not have extras directory. Is tomcat-juli-adapters file no longer required? What will be the best way to configure tomcat logs using log4j. My main interest is uploading access logs and catalina.out to AWS/S3 bucket. For our application logs, we are already doing this. If there is a better way to achieve this, open to that solution too. Regards, Niranjan - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
No need to be discouraged. Docker is just a set of tools. You can still use docker to create images, but you dont need docker to use those images in a container. K8s is using industry standard containerd. https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/ John Larsen On Mon, Jun 28, 2021 at 2:22 PM Eric Robinson wrote: > Guido, > > I think you intended that message for me, not Brian. Thanks much for the > feedback. I have been reading about Kubernetes, but I got discouraged when > I saw that they dropped Docker support, since Docker seems to be the most > popular containeriziation technology. Also, most of the Kubernetes > tutorials I saw on YouTube seem to approach it as a dev platform, and we're > not developers. > > -Eric > > > > -Original Message- > > From: Guido Jäkel > > Sent: Monday, June 28, 2021 2:43 PM > > To: Brian Wolfe > > Cc: Tomcat Users List > > Subject: Re: 500 instances of tomcat on the same server > > > > Dear Brian, > > > > please take the time to read about Linux Kernel namespaces as the > technical > > base of containers. It's like two viewpoints to one thing. Take the > network > > namespace as an example: From the conceptual point of view it looks like > > you have N indipended, functional identical "IP Stacks". But from the > > technical point of view, it's just the "well known" single instance just > with an > > additional field at all items that need this (packets, routing tables, > ...) to take > > a tag value that identify the namespace instance. > > > > You may use namespaces with the raw tools like enterns or with LXC or > > Dockers. During runtime of a started container, there's nothing more you > > have to trust but the kernel because for the basics, there's no need of > > additional userland processes to keep a container running. > > > > To run an application in a "container", you start it with a bunch of > instances of > > this namespaces, at least the process namespace. You'll probably take the > > same name for the technical namespace instances - from the conceptual > > point of view this is the name of the container. > > > > Most will start something like the init binary located in a directory > tree of a > > small Linux distribution userland. This may "boot" common services and > the > > result may act like an "indipended platfrom". But you may also launch > just > > single high-level applications like a JVM running a Tomcat. > > > > That's very close to your architecture, but much more easy to handle. > For the > > network stack e.g. you may use the same ports for listeners and have the > full > > range of ports available for connections in each namespace. There are > > different ways available to route the traffic, but in any case you may > use > > individual IPs in each namespace. > > > > greetings > > > > Guido > > > > On 2021-06-28 19:22, Brian Wolfe wrote: > > > Generally, I'd agree too. We are considering using containers, but I'm > > > not yet sure what that buys us in terms of stability. > > > > - > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: users-h...@tomcat.apache.org > > Disclaimer : This email and any files transmitted with it are confidential > and intended solely for intended recipients. If you are not the named > addressee you should not disseminate, distribute, copy or alter this email. > Any views or opinions presented in this email are solely those of the > author and might not represent those of Physician Select Management. > Warning: Although Physician Select Management has taken reasonable > precautions to ensure no viruses are present in this email, the company > cannot accept responsibility for any loss or damage arising from the use of > this email or attachments. >
RE: 500 instances of tomcat on the same server
Guido, I think you intended that message for me, not Brian. Thanks much for the feedback. I have been reading about Kubernetes, but I got discouraged when I saw that they dropped Docker support, since Docker seems to be the most popular containeriziation technology. Also, most of the Kubernetes tutorials I saw on YouTube seem to approach it as a dev platform, and we're not developers. -Eric > -Original Message- > From: Guido Jäkel > Sent: Monday, June 28, 2021 2:43 PM > To: Brian Wolfe > Cc: Tomcat Users List > Subject: Re: 500 instances of tomcat on the same server > > Dear Brian, > > please take the time to read about Linux Kernel namespaces as the technical > base of containers. It's like two viewpoints to one thing. Take the network > namespace as an example: From the conceptual point of view it looks like > you have N indipended, functional identical "IP Stacks". But from the > technical point of view, it's just the "well known" single instance just with > an > additional field at all items that need this (packets, routing tables, ...) > to take > a tag value that identify the namespace instance. > > You may use namespaces with the raw tools like enterns or with LXC or > Dockers. During runtime of a started container, there's nothing more you > have to trust but the kernel because for the basics, there's no need of > additional userland processes to keep a container running. > > To run an application in a "container", you start it with a bunch of > instances of > this namespaces, at least the process namespace. You'll probably take the > same name for the technical namespace instances - from the conceptual > point of view this is the name of the container. > > Most will start something like the init binary located in a directory tree of > a > small Linux distribution userland. This may "boot" common services and the > result may act like an "indipended platfrom". But you may also launch just > single high-level applications like a JVM running a Tomcat. > > That's very close to your architecture, but much more easy to handle. For the > network stack e.g. you may use the same ports for listeners and have the full > range of ports available for connections in each namespace. There are > different ways available to route the traffic, but in any case you may use > individual IPs in each namespace. > > greetings > > Guido > > On 2021-06-28 19:22, Brian Wolfe wrote: > > Generally, I'd agree too. We are considering using containers, but I'm > > not yet sure what that buys us in terms of stability. > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
RE: 500 instances of tomcat on the same server
> -Original Message- > From: Brian Wolfe > Sent: Monday, June 28, 2021 12:23 PM > To: Tomcat Users List > Subject: Re: 500 instances of tomcat on the same server > > I tend to agree with the initial assessment from Mark, your only issue would > be on the OS level. # of file descriptors for connections. That many tomcat > servers and your gonna start using a lot of ports and push the OS limits on > file > read/write capabilities. > Those are some of my concerns as well, which is why I asked the question. I can work around the limitation on ephemeral client ports by adding additional IPs to the box and using the localSocketAddress property of Connector/J. I can set the max file descriptors to something like half a million. Are there other potential limitations you can think off? > From an architecture perspective you should probably work on moving to a > more modern deployment model of containerization of these apps. You > would be better served by containerizing each customer deployment and > running them on a kubernetes cluster. you can avoid the need for having > large machines and scale more appropriately. and moving between hardware > would be as simple as adding/removing nodes to your cluster. It is 2 cents well spent. I have also considered Kubernetes and containerization, but I don't yet understand it well enough to know exactly how it benefits me. >It sounds like > the apps must be simple to be able to scale it to different clients like that. > just my 2 cents. > Not simple, but predictable. We've been hosting it for over decade, and we have a good feel for its resource utilization. -Eric > On Mon, Jun 28, 2021 at 1:12 PM Eric Robinson > wrote: > > > > > > > > > > > > -Original Message- > > > From: Mark Thomas > > > Sent: Monday, June 28, 2021 9:04 AM > > > To: users@tomcat.apache.org > > > Subject: Re: 500 instances of tomcat on the same server > > > > > > On 28/06/2021 14:53, Christopher Schultz wrote: > > > > Eric, > > > > > > > > On 6/25/21 22:58, Eric Robinson wrote: > > > >> We can run 75 to 125 instances of tomcat on a single Linux server > > > >> with > > > >> 12 cores and 128GB RAM. It works great. CPU is around 25%, our > > > >> JVMs are not throwing OOMEs, iowait is minimal, and network > > > >> traffic is about 30Mbps. We're happy with the results. > > > >> > > > >> Now we're upping the ante. We have a 48-core server with 1TB RAM, > > > >> and we're planning to run 600+ tomcat instances on it simultaneously. > > > >> What caveats or pitfalls should we watch out for? Are there any > > > >> hard limits that would prevent this from working as expected? > > > > If you have the resources, I see no reason why this would present > > > > any problems. > > > > > > > > On the other hand, what happens when you need to upgrade the OS > on > > > > this beast? You are now talking about disturbing not 72-125 > > > > clients, but 600 of them. > > > > > > > > If I had a beast like this, I'd run VMWare (or similar) on it, > > > > carve it up into virtual machines, and run fewer clients on > > > > each just for the sheer flexibility of it. > > > That just moves the goal posts. You'll have the same issue when the > > > hypervisor needs updating (which admittedly may need a reboot less > > > often than the OS). > > > > > > > If this is already a virtualized/cloud environment, then I think > > > > you're doing it wrong: don't provision one huge instance and use > > > > it for multiple clients. Instead, provision lots of small > > > > instances and use them for fewer (or even 1) at a time. > > > > > > But it adds the overhead of an OS for each instance. And costs if > > > you > > have to > > > pay for that OS instance. > > > > > > > The overhead issue is an important factor. The other is the fact that > > it's a canned app, supported by the publisher, and doing it our way > > pays big dividends in terms of that workflow. > > > > > As always there are trade-offs to be made and the "right" answer > > > will > > vary > > > based on circumstances and what you are trying to optimize for. I do > > agree > > > that, generally, more smaller instances will be a closer fit to more > > > use > > cases > > > but that is only a general answer. > > > > > > > Generally, I'd agree too. We are considering using containers, but I'm > > not yet sure what that buys us in terms of stability. > > > > > Mark > > > > > > > > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > Disclaimer : This email and any files transmitted with it are > > confidential and intended solely for intended recipients. If you are > > not the named addressee you should not disseminate, distribute, copy or > alter this email. > > Any views or opinions presented in this email are solely those of the > > author and might not represent those of Physician Select Manag
Re: 500 instances of tomcat on the same server
Dear Brian, please take the time to read about Linux Kernel namespaces as the technical base of containers. It's like two viewpoints to one thing. Take the network namespace as an example: From the conceptual point of view it looks like you have N indipended, functional identical "IP Stacks". But from the technical point of view, it's just the "well known" single instance just with an additional field at all items that need this (packets, routing tables, ...) to take a tag value that identify the namespace instance. You may use namespaces with the raw tools like enterns or with LXC or Dockers. During runtime of a started container, there's nothing more you have to trust but the kernel because for the basics, there's no need of additional userland processes to keep a container running. To run an application in a "container", you start it with a bunch of instances of this namespaces, at least the process namespace. You'll probably take the same name for the technical namespace instances - from the conceptual point of view this is the name of the container. Most will start something like the init binary located in a directory tree of a small Linux distribution userland. This may "boot" common services and the result may act like an "indipended platfrom". But you may also launch just single high-level applications like a JVM running a Tomcat. That's very close to your architecture, but much more easy to handle. For the network stack e.g. you may use the same ports for listeners and have the full range of ports available for connections in each namespace. There are different ways available to route the traffic, but in any case you may use individual IPs in each namespace. greetings Guido On 2021-06-28 19:22, Brian Wolfe wrote: > Generally, I'd agree too. We are considering using containers, but I'm not > yet sure what that buys us in terms of stability. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Possible bug in http2 window size handling in tomcat 9.0.45
On 28/06/2021 15:11, Mark Thomas wrote: On 28/06/2021 10:53, Erik Nilsson wrote: Yep, something seems to go wrong with the waitingFor field in WindowAllocationManager. We are developing a quite complex embedded cms application, don't know if I will be able to share this application. Hopefully you can reproduce this anyway by using the nghttp client and another large webapp? One thing to notice is that we can't reproduce this by accessing the webapplication directly with any browser, but with a f5-proxy in production and nghttp it occurs almost every time. I'll see what I can do to reproduce this. Any chance you could run Tomcat with http2 debug logging enabled, trigger the issue and then post the debug logs (or send them to me privately if you prefer?). I've found a bug in the sendfile handling. Can you add useSendfile="false" to the associated element and see if that works around the issue. Thanks, Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
I tend to agree with the initial assessment from Mark, your only issue would be on the OS level. # of file descriptors for connections. That many tomcat servers and your gonna start using a lot of ports and push the OS limits on file read/write capabilities. >From an architecture perspective you should probably work on moving to a more modern deployment model of containerization of these apps. You would be better served by containerizing each customer deployment and running them on a kubernetes cluster. you can avoid the need for having large machines and scale more appropriately. and moving between hardware would be as simple as adding/removing nodes to your cluster. It sounds like the apps must be simple to be able to scale it to different clients like that. just my 2 cents. On Mon, Jun 28, 2021 at 1:12 PM Eric Robinson wrote: > > > > > > -Original Message- > > From: Mark Thomas > > Sent: Monday, June 28, 2021 9:04 AM > > To: users@tomcat.apache.org > > Subject: Re: 500 instances of tomcat on the same server > > > > On 28/06/2021 14:53, Christopher Schultz wrote: > > > Eric, > > > > > > On 6/25/21 22:58, Eric Robinson wrote: > > >> We can run 75 to 125 instances of tomcat on a single Linux server > > >> with > > >> 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs > > >> are not throwing OOMEs, iowait is minimal, and network traffic is > > >> about 30Mbps. We're happy with the results. > > >> > > >> Now we're upping the ante. We have a 48-core server with 1TB RAM, and > > >> we're planning to run 600+ tomcat instances on it simultaneously. > > >> What caveats or pitfalls should we watch out for? Are there any hard > > >> limits that would prevent this from working as expected? > > > If you have the resources, I see no reason why this would present any > > > problems. > > > > > > On the other hand, what happens when you need to upgrade the OS on > > > this beast? You are now talking about disturbing not 72-125 clients, > > > but 600 of them. > > > > > > If I had a beast like this, I'd run VMWare (or similar) on it, carve > > > it up into virtual machines, and run fewer clients on each just > > > for the sheer flexibility of it. > > That just moves the goal posts. You'll have the same issue when the > > hypervisor needs updating (which admittedly may need a reboot less often > > than the OS). > > > > > If this is already a virtualized/cloud environment, then I think > > > you're doing it wrong: don't provision one huge instance and use it > > > for multiple clients. Instead, provision lots of small instances and > > > use them for fewer (or even 1) at a time. > > > > But it adds the overhead of an OS for each instance. And costs if you > have to > > pay for that OS instance. > > > > The overhead issue is an important factor. The other is the fact that it's > a canned app, supported by the publisher, and doing it our way pays big > dividends in terms of that workflow. > > > As always there are trade-offs to be made and the "right" answer will > vary > > based on circumstances and what you are trying to optimize for. I do > agree > > that, generally, more smaller instances will be a closer fit to more use > cases > > but that is only a general answer. > > > > Generally, I'd agree too. We are considering using containers, but I'm not > yet sure what that buys us in terms of stability. > > > Mark > > > > - > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: users-h...@tomcat.apache.org > > Disclaimer : This email and any files transmitted with it are confidential > and intended solely for intended recipients. If you are not the named > addressee you should not disseminate, distribute, copy or alter this email. > Any views or opinions presented in this email are solely those of the > author and might not represent those of Physician Select Management. > Warning: Although Physician Select Management has taken reasonable > precautions to ensure no viruses are present in this email, the company > cannot accept responsibility for any loss or damage arising from the use of > this email or attachments. > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > -- Thanks, Brian Wolfe https://www.linkedin.com/in/brian-wolfe-3136425a/
RE: 500 instances of tomcat on the same server
> -Original Message- > From: Mark Thomas > Sent: Monday, June 28, 2021 9:04 AM > To: users@tomcat.apache.org > Subject: Re: 500 instances of tomcat on the same server > > On 28/06/2021 14:53, Christopher Schultz wrote: > > Eric, > > > > On 6/25/21 22:58, Eric Robinson wrote: > >> We can run 75 to 125 instances of tomcat on a single Linux server > >> with > >> 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs > >> are not throwing OOMEs, iowait is minimal, and network traffic is > >> about 30Mbps. We're happy with the results. > >> > >> Now we're upping the ante. We have a 48-core server with 1TB RAM, and > >> we're planning to run 600+ tomcat instances on it simultaneously. > >> What caveats or pitfalls should we watch out for? Are there any hard > >> limits that would prevent this from working as expected? > > If you have the resources, I see no reason why this would present any > > problems. > > > > On the other hand, what happens when you need to upgrade the OS on > > this beast? You are now talking about disturbing not 72-125 clients, > > but 600 of them. > > > > If I had a beast like this, I'd run VMWare (or similar) on it, carve > > it up into virtual machines, and run fewer clients on each just > > for the sheer flexibility of it. > That just moves the goal posts. You'll have the same issue when the > hypervisor needs updating (which admittedly may need a reboot less often > than the OS). > > > If this is already a virtualized/cloud environment, then I think > > you're doing it wrong: don't provision one huge instance and use it > > for multiple clients. Instead, provision lots of small instances and > > use them for fewer (or even 1) at a time. > > But it adds the overhead of an OS for each instance. And costs if you have to > pay for that OS instance. > The overhead issue is an important factor. The other is the fact that it's a canned app, supported by the publisher, and doing it our way pays big dividends in terms of that workflow. > As always there are trade-offs to be made and the "right" answer will vary > based on circumstances and what you are trying to optimize for. I do agree > that, generally, more smaller instances will be a closer fit to more use cases > but that is only a general answer. > Generally, I'd agree too. We are considering using containers, but I'm not yet sure what that buys us in terms of stability. > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: 500 instances of tomcat on the same server
> -Original Message- > From: Christopher Schultz > Sent: Monday, June 28, 2021 8:54 AM > To: users@tomcat.apache.org > Subject: Re: 500 instances of tomcat on the same server > > Eric, > > On 6/25/21 22:58, Eric Robinson wrote: > > We can run 75 to 125 instances of tomcat on a single Linux server with > > 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs > > are not throwing OOMEs, iowait is minimal, and network traffic is > > about 30Mbps. We're happy with the results. > > > > Now we're upping the ante. We have a 48-core server with 1TB RAM, and > > we're planning to run 600+ tomcat instances on it simultaneously. > > What caveats or pitfalls should we watch out for? Are there any hard > > limits that would prevent this from working as expected? > If you have the resources, I see no reason why this would present any > problems. > > On the other hand, what happens when you need to upgrade the OS on this > beast? You are now talking about disturbing not 72-125 clients, but 600 of > them. > There are two load-balanced servers, each with adequate power to support the whole load. When we want to maintain Server A, we drain it at the load balancer and wait for the last active connection to complete. Then we reboot/maintain the server and add it back into the rotation gracefully. > If I had a beast like this, I'd run VMWare (or similar) on it, carve it up > into > virtual machines, and run fewer clients on each just for the sheer > flexibility > of it. > We considered doing it that way. Performance is top priority, so we ultimately decided to run the instances on metal rather than introducing a few trillion lines of OS code into the mix. We might consider containerizing. > If this is already a virtualized/cloud environment, then I think you're doing > it > wrong: don't provision one huge instance and use it for multiple clients. > Instead, provision lots of small instances and use them for fewer (or even 1) > at a time. > That makes sense until you know the environment better. It's a canned application and we're not the publisher. Breaking it out this way gives us the ability to present each customer and a unique entity to the publisher for support purposes. When their techs connect, the sandbox allows them to troubleshoot and support our mutual customer independently, which puts them in an environment their techs are comfortable with, and removed the risk of them doing something that impacts everybody on the server (or in the VM, if we used those). All I can tell you is we've been running it this way for 15 years and we've never looked back and wished we were doing it differently. > -chris > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: [Possible Spam] Re: TLSv1.3 Support in Tomcat
I am not sure how it is not working for you, so this may not be relevant. However, this caused me a lot of confusion. https://stackoverflow.com/questions/57601284/java-11-and-12-ssl-sockets-fail-on-a-handshake-failure-error-with-tlsv1-3-enable I had to disable TLS 1.3 to get my Java client to connect to Tomcat. (I had a thread "Strange connection error" and "[Possible Spam] Re: Strange connection error" starting on June 10 or so.) Good luck, Mark Claassen Senior Software Engineer Donnell Systems, Inc. 130 South Main Street Leighton Plaza Suite 375 South Bend, IN 46601 E-mail: mailto:mclaas...@ocie.net Voice: (574)232-3784 Fax: (574)232-4014 Disclaimer: The opinions provided herein do not necessarily state or reflect those of Donnell Systems, Inc.(DSI). DSI makes no warranty for and assumes no legal liability or responsibility for the posting. -Original Message- From: Christopher Schultz Sent: Monday, June 28, 2021 9:57 AM To: users@tomcat.apache.org Subject: [Possible Spam] Re: TLSv1.3 Support in Tomcat Importance: Low Abirami, On 6/28/21 07:16, S Abirami wrote: > TLSv1.3 support is available in Tomcat. > > I tried just updating server.xml[sslEnabledProtocols=TLSv1.3] and > restarted tomcat. It doesn't work. > > [We are using Tomcat 9.0.46 and JDK 8u291] > > Please let me know any other configuration also needs to be changed. Can you please post your configuration (minus any secrets)? When you say "it doesn't work", what exactly do you mean? -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Questions about Integrated Windows Authentication
Sorry Mark, I've clicked the wrong button in my mail client :( On 28.06.2021 15:29, Mark Thomas wrote: Note that Tomcat 7 is no longer supported. I guess it's nearly the same for all versions of Tomcat. That looks more like some form of configuration issue but I always found the Kerberos error message rather hard to decipher. AFAIK, the Kerberos is working fine. This error occurs in JNDIRealm's getPrincipal method. One log line before, Kerberos reports Found ticket for HTTP/apps.atlas-03t.gvsn.local@GVSN.LOCAL to go to krbtgt/GVSN.LOCAL@GVSN.LOCAL expiring on Thu Jun 24 18:26:05 CEST 2021 So, there is a ticket. However, JNDIRealm cannot use it or the ticket does not allow binding to the directory with that user. I'm not understanding the whole process, so I was asking if someone has more glue on that. 2. Fallback Authenticator It has been mentioned before. There is this on the Wiki: https://cwiki.apache.org/confluence/display/TOMCAT/SSLWithFORMFallback Will have a look at that. It's basically what I was thinking about adding a fallback to SpnegoAuthenticator only. As with most enhancements, whether it is accepted is going to depend largely on the benefit it brings vs how complex / invasive the code is. For sure. Rémy mentioned he was looking for a development project. Maybe this could be it. I guess, Rémy was taking my user attributes Realm extension as development project... You might be able to authenticate external users in a reverse proxy and have it pass the user ID to Tomcat rather than have Tomcat do the authentication. I read about that somewhere some months ago. However, I don't know how to get the authentication from the reverse proxy (my Tomcat already runs behind an Apache HTTPD using mod_proxy_ajp) to Tomcat? Finally, Tomcat needs the Principal and a couple of roles for authorization (including my additional user attributes). Passing the user ID only is likely not sufficient. Could you please describe that in more detail or point me to some sites to learn more about that? - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
Mark, On 6/28/21 10:04, Mark Thomas wrote: On 28/06/2021 14:53, Christopher Schultz wrote: Eric, On 6/25/21 22:58, Eric Robinson wrote: We can run 75 to 125 instances of tomcat on a single Linux server with 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs are not throwing OOMEs, iowait is minimal, and network traffic is about 30Mbps. We're happy with the results. Now we're upping the ante. We have a 48-core server with 1TB RAM, and we're planning to run 600+ tomcat instances on it simultaneously. What caveats or pitfalls should we watch out for? Are there any hard limits that would prevent this from working as expected? If you have the resources, I see no reason why this would present any problems. On the other hand, what happens when you need to upgrade the OS on this beast? You are now talking about disturbing not 72-125 clients, but 600 of them. If I had a beast like this, I'd run VMWare (or similar) on it, carve it up into virtual machines, and run fewer clients on each just for the sheer flexibility of it. > That just moves the goal posts. You'll have the same issue when the hypervisor needs updating (which admittedly may need a reboot less often than the OS). Agreed. My assertion is that the hypervisor should require updates far less frequently. My AWS EC2 instances can stay up for months without being forcibly rebooted, for example. (I tend to restart them, anyway, for e.g. OS updates, but the point is the same.) If this is already a virtualized/cloud environment, then I think you're doing it wrong: don't provision one huge instance and use it for multiple clients. Instead, provision lots of small instances and use them for fewer (or even 1) at a time. But it adds the overhead of an OS for each instance. And costs if you have to pay for that OS instance. As always there are trade-offs to be made and the "right" answer will vary based on circumstances and what you are trying to optimize for. I do agree that, generally, more smaller instances will be a closer fit to more use cases but that is only a general answer. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Possible bug in http2 window size handling in tomcat 9.0.45
On 28/06/2021 10:53, Erik Nilsson wrote: Yep, something seems to go wrong with the waitingFor field in WindowAllocationManager. We are developing a quite complex embedded cms application, don't know if I will be able to share this application. Hopefully you can reproduce this anyway by using the nghttp client and another large webapp? One thing to notice is that we can't reproduce this by accessing the webapplication directly with any browser, but with a f5-proxy in production and nghttp it occurs almost every time. I'll see what I can do to reproduce this. Any chance you could run Tomcat with http2 debug logging enabled, trigger the issue and then post the debug logs (or send them to me privately if you prefer?). Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
On 28/06/2021 14:53, Christopher Schultz wrote: Eric, On 6/25/21 22:58, Eric Robinson wrote: We can run 75 to 125 instances of tomcat on a single Linux server with 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs are not throwing OOMEs, iowait is minimal, and network traffic is about 30Mbps. We're happy with the results. Now we're upping the ante. We have a 48-core server with 1TB RAM, and we're planning to run 600+ tomcat instances on it simultaneously. What caveats or pitfalls should we watch out for? Are there any hard limits that would prevent this from working as expected? If you have the resources, I see no reason why this would present any problems. On the other hand, what happens when you need to upgrade the OS on this beast? You are now talking about disturbing not 72-125 clients, but 600 of them. If I had a beast like this, I'd run VMWare (or similar) on it, carve it up into virtual machines, and run fewer clients on each just for the sheer flexibility of it. That just moves the goal posts. You'll have the same issue when the hypervisor needs updating (which admittedly may need a reboot less often than the OS). If this is already a virtualized/cloud environment, then I think you're doing it wrong: don't provision one huge instance and use it for multiple clients. Instead, provision lots of small instances and use them for fewer (or even 1) at a time. But it adds the overhead of an OS for each instance. And costs if you have to pay for that OS instance. As always there are trade-offs to be made and the "right" answer will vary based on circumstances and what you are trying to optimize for. I do agree that, generally, more smaller instances will be a closer fit to more use cases but that is only a general answer. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: TLSv1.3 Support in Tomcat
Abirami, On 6/28/21 07:16, S Abirami wrote: TLSv1.3 support is available in Tomcat. I tried just updating server.xml[sslEnabledProtocols=TLSv1.3] and restarted tomcat. It doesn't work. [We are using Tomcat 9.0.46 and JDK 8u291] Please let me know any other configuration also needs to be changed. Can you please post your configuration (minus any secrets)? When you say "it doesn't work", what exactly do you mean? -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
Eric, On 6/25/21 22:58, Eric Robinson wrote: We can run 75 to 125 instances of tomcat on a single Linux server with 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs are not throwing OOMEs, iowait is minimal, and network traffic is about 30Mbps. We're happy with the results. Now we're upping the ante. We have a 48-core server with 1TB RAM, and we're planning to run 600+ tomcat instances on it simultaneously. What caveats or pitfalls should we watch out for? Are there any hard limits that would prevent this from working as expected? If you have the resources, I see no reason why this would present any problems. On the other hand, what happens when you need to upgrade the OS on this beast? You are now talking about disturbing not 72-125 clients, but 600 of them. If I had a beast like this, I'd run VMWare (or similar) on it, carve it up into virtual machines, and run fewer clients on each just for the sheer flexibility of it. If this is already a virtualized/cloud environment, then I think you're doing it wrong: don't provision one huge instance and use it for multiple clients. Instead, provision lots of small instances and use them for fewer (or even 1) at a time. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Re-Use TCP Source Ports if the Socket is Unique?
Eric, On 6/25/21 22:09, Eric Robinson wrote: -Original Message- From: Olaf Kock Sent: Friday, June 25, 2021 8:07 AM To: users@tomcat.apache.org Subject: Re: Re-Use TCP Source Ports if the Socket is Unique? On 25.06.21 14:46, Eric Robinson wrote: Olaf and Scott -- Thanks to both of you for your comments. I may have asked my question poorly, since what you both described is the way I understand TCP to work. There is no correlation between an incoming connection to tomcat and its outgoing connection to a database backend, nor would I expect there to be. Perhaps a simpler way to ask my question is: when a server has multiple IPs, which one does tomcat use as its source IP when it initiates a three-way handshake with a remote machine? For example, suppose my server has IP addresses 10.0.0.1 and 10.0.0.2, and my tomcat connector looks like this... Tomcat is now listening on IP 10.0.0.2. But here's the question. If tomcat needs to initiate a TCP session to a remote machine (acting as a TCP client), will it use 10.0.0.1 or 10.0.0.2 as the source IP of the outbound connection? I'm assuming it will use the same IP that the connector is configured to listen on. Hi Eric, again: There's no correlation. Your question boils down to a context-free "which source IP does tomcat use for outgoing connections?". In fact, Tomcat doesn't use any. It just asks the runtime environment (ultimately I'd expect the OS) for a connection to a particular destination, then it uses that. How the connection is then established will depend on * available network adapters * best route to the target address * OS or network configuration It will /not/ depend on any of Tomcat's Connector-configurations whatsoever Got it. Then, given a tomcat server with one NIC and two IP addresses, 10.0.0.2 and 10.0.0.3, when tomcat connects to a server on the same subnet at 10.0.0.50, what logic does the OS use to select the source IP, all else being equal? Obviously neither IP has a routing advantage. You are confusing NIC and interface. A NIC is a piece of hardware, while an interface is a software concept. Often, they are 1:1 so you have 1 NIC and 1 interface. But if you have two interfaces, they must have something which differentiates them, or the network configuration will be ... insane. I'm sure it's possible to get a pair of network interfaces configured with both of them on the same network segment, but the IP stack will surely have rules for deciding which interface is used for the (client) socket by default if you don't specify one. But those rules are beyond Tomcat, beyond the JVM, and rest with the OS itself. So the answer is "it depends" and "if you care, you should specify which one you want." -chris -Original Message- From: Olaf Kock Sent: Friday, June 25, 2021 3:01 AM To: users@tomcat.apache.org Subject: Re: Re-Use TCP Source Ports if the Socket is Unique? On 25.06.21 05:19, Eric Robinson wrote: Thanks for the feedback, Daniel. I guess the answer depends on whether the socket libraries use the tomcat listening port as the source IP. If you have three tomcat instances listening on three different IPs, each instance should be able to open a client connection using the same source port, as long as each tomcat uses its listening IP as the source IP of the socket. That's the part I'm still not sure about. My expectation is that database connections do not have any correlation with the listening port: Technically, DB connection pools can be shared across all contained Hosts and Connectors /within a single tomcat/, and when multiple processes are added to the game, it doesn't really change anything. In fact, it's not uncommon that there's a public facing network adapter, where a http-connector listens, but a completely different network adapter for any backend communication - e.g. to the database. All that I expect a database driver to do is to specify where it wants to connect to, and the OS figures out how that connection needs to be routed. That's utterly independent of any http connection that comes in to the same process. So: Don't expect any correlation, and you're safe. (Note: There /may/ be ways to configure a db-driver to specify a source address, but I'd expect that rather to add a potential failure rather than anything that I'd want to control. If you interpret such a situation differently: Please elaborate) Best, Olaf - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Sele
Re: Questions about Integrated Windows Authentication
On 28/06/2021 10:36, Carsten Klein wrote: Hi there, I have two questions about Tomcat's Integrated Windows Authentication: Tomcat is stuck on version 7.0.52 on an outdated Ubuntu 14.04 LTS. Note that Tomcat 7 is no longer supported. 1. useDelegatedCredential = true I'm using JNDIRealm together with the SPNEGO authenticator. If the Realm's option 'useDelegatedCredential' is set to true, I'm getting exception: javax.naming.AuthenticationException: GSSAPI [Root exception is javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]]; remaining name 'xxx.yyy.zzz...' Everything works fine when not using delegated credentials, but configured connectionName and connectionPassword. What's the reason for that? Is it a Tomcat configuration issue? Or, is the client (Google Chrome) not sending enough (credential?) information: There is Chrome's Policy option 'AuthNegotiateDelegateWhitelist' (deprecated, replaced by 'AuthNegotiateDelegateAllowlist') which must be configured in order to delegate the user's identity. However, setting any of these policy settings to true does not help. See https://www.chromium.org/developers/design-documents/http-authentication Is it a limitation/setting in Active Directory Server? The exception occurs after SpnegoAuthenticator has contacted ADS trough Kerberos. Is the response obtained from that Kerberos call not suitable for using delegated credentials? That looks more like some form of configuration issue but I always found the Kerberos error message rather hard to decipher. 2. Fallback Authenticator We have some clients connected through VPN, whose Windows sessions are not logged on to the Active Directory's Windows Domain, so Integrated Windows Authentication cannot work. SpnegoAuthenticator reports 'No authorization header sent by client'. However, I've not yet found a way to fall back to e. g. FORM authentication for those clients. AFAIK, there is no way to do this with Tomcat, since a Context can only have one single authenticator valve. Oracle's WebLogic Server support configuring more than one authentication method, by adding something like CLIENT-CERT,BASIC into web.xml. What about adding support for that or something similar in Tomcat? A CombinedAuthenticator (like with CombinedRealm) could be a solution. That could instantiate other required Authenticator valves and pass the request from one to the other until authentication succeeds. Those valves must not necessarily be queued in the container's pipeline, but could be called by the CombinedAuthenticator valve. Thats likely not too simple but it could be done. Are you open to such a solution? It has been mentioned before. There is this on the Wiki: https://cwiki.apache.org/confluence/display/TOMCAT/SSLWithFORMFallback As with most enhancements, whether it is accepted is going to depend largely on the benefit it brings vs how complex / invasive the code is. Rémy mentioned he was looking for a development project. Maybe this could be it. I believe that only the SpnegoAuthenticator needs such a fallback, as it uses mechanisms that you can't just add to any client. (In contrast, you could always distribute a X509 certificate or use FORM, BASIC, or DIGEST login. But you can't add your client to a Windows Domain just in order to log in to an application.) Adding a fallback mechanism to SpnegoAuthenticator only may be much easier. What other solutions do you know? You might be able to authenticate external users in a reverse proxy and have it pass the user ID to Tomcat rather than have Tomcat do the authentication. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: TLSv1.3 Support in Tomcat
Hi All, We are using Tomcat 9.0.46 and JDK 8u291 Regards, Abirami.S -Original Message- From: S Abirami Sent: Monday, June 28, 2021 4:47 PM To: Tomcat Users List Subject: TLSv1.3 Support in Tomcat Hi All, TLSv1.3 support is available in Tomcat. I tried just updating server.xml[sslEnabledProtocols=TLSv1.3] and restarted tomcat. It doesn't work. Please let me know any other configuration also needs to be changed. Regards, Abirami.S - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: TLSv1.3 Support in Tomcat
On Mon, Jun 28, 2021, 06:17 S Abirami wrote: > Hi All, > > TLSv1.3 support is available in Tomcat. > > I tried just updating server.xml[sslEnabledProtocols=TLSv1.3] and > restarted tomcat. It doesn't work. > > Please let me know any other configuration also needs to be changed. > You did not mention the version of Tomcat and Java you are using.
TLSv1.3 Support in Tomcat
Hi All, TLSv1.3 support is available in Tomcat. I tried just updating server.xml[sslEnabledProtocols=TLSv1.3] and restarted tomcat. It doesn't work. Please let me know any other configuration also needs to be changed. Regards, Abirami.S
Re: Possible bug in http2 window size handling in tomcat 9.0.45
Yep, something seems to go wrong with the waitingFor field in WindowAllocationManager. We are developing a quite complex embedded cms application, don't know if I will be able to share this application. Hopefully you can reproduce this anyway by using the nghttp client and another large webapp? One thing to notice is that we can't reproduce this by accessing the webapplication directly with any browser, but with a f5-proxy in production and nghttp it occurs almost every time. /Erik Den mån 28 juni 2021 kl 10:52 skrev Mark Thomas : > On 27/06/2021 12:05, Erik Nilsson wrote: > > We might have found an issue with the window size in http2 in Tomcat > 9.0.45. > > Thanks for the heads up. > > 9.0.45 has fixes for all the known issues with window size management so > this looks like a potential new bug. > > > java.lang.IllegalStateException: Connection [483], Stream [21], Already > waiting > > at > org.apache.coyote.http2.WindowAllocationManager.waitFor(WindowAllocationManager.java:143) > > at > org.apache.coyote.http2.WindowAllocationManager.waitForConnection(WindowAllocationManager.java:84) > > at > org.apache.coyote.http2.Stream.waitForConnectionAllocation(Stream.java:257) > > at > org.apache.coyote.http2.Http2UpgradeHandler.reserveWindowSize(Http2UpgradeHandler.java:903) > > at > org.apache.coyote.http2.Stream$StreamOutputBuffer.flush(Stream.java:904) > > at > org.apache.coyote.http2.Stream$StreamOutputBuffer.doWrite(Stream.java:823) > > at > org.apache.coyote.http2.Http2OutputBuffer.doWrite(Http2OutputBuffer.java:59) > > at org.apache.coyote.Response.doWrite(Response.java:606) > > at > org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:340) > > at > org.apache.catalina.connector.OutputBuffer.appendByteArray(OutputBuffer.java:753) > > at > org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:682) > > at > org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:388) > > at > org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:366) > > at > org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:96) > > at > org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89) > > > > > > This problem occurs when we are using the nghttp client to receive > > data from our root page (nghttp -vnsay https://localhost). By > > decreasing the initial window size to 8 kb in nghttp (flag -w 13), the > > problem disappears. Another solution is to set the http2-connector > > attribute maxConcurrentStreamExecution="1". > > Hmm. That suggests some sort of concurrency issue. > > How easy is this to reproduce? I'll see if I can reproduce this locally > but it may well depend on the structure of the page you are testing with. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > --
Questions about Integrated Windows Authentication
Hi there, I have two questions about Tomcat's Integrated Windows Authentication: Tomcat is stuck on version 7.0.52 on an outdated Ubuntu 14.04 LTS. 1. useDelegatedCredential = true I'm using JNDIRalm together with the SPNEGO authenticator. If the Realm's option 'useDelegatedCredential' is set to true, I'm getting exception: javax.naming.AuthenticationException: GSSAPI [Root exception is javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]]; remaining name 'xxx.yyy.zzz...' Everything works fine when not using delegated credentials, but configured connectionName and connectionPassword. What's the reason for that? Is it a Tomcat configuration issue? Or, is the client (Google Chrome) not sending enough (credential?) information: There is Chrome's Policy option 'AuthNegotiateDelegateWhitelist' (deprecated, replaced by 'AuthNegotiateDelegateAllowlist') which must be configured in order to delegate the user's identity. However, setting any of these policy settings to true does not help. See https://www.chromium.org/developers/design-documents/http-authentication Is it a limitation/setting in Active Directory Server? The exception occurs after SpnegoAuthenticator has contacted ADS trough Kerberos. Is the response obtained from that Kerberos call not suitable for using delegated credentials? 2. Fallback Authenticator We have some clients connected through VPN, whose Windows sessions are not logged on to the Active Directory's Windows Domain, so Integrated Windows Authentication cannot work. SpnegoAuthenticator reports 'No authorization header sent by client'. However, I've not yet found a way to fall back to e. g. FORM authentication for those clients. AFAIK, there is no way to do this with Tomcat, since a Context can only have one single authenticator valve. Oracle's WebLogic Server support configuring more than one authentication method, by adding something like CLIENT-CERT,BASIC into web.xml. What about adding support for that or something similar in Tomcat? A CombinedAuthenticator (like with CombinedRealm) could be a solution. That could instantiate other required Authenticator valves and pass the request from one to the other until authentication succeeds. Those valves must not necessarily be queued in the container's pipeline, but could be called by the CombinedAuthenticator valve. Thats likely not too simple but it could be done. Are you open to such a solution? I believe that only the SpnegoAuthenticator needs such a fallback, as it uses mechanisms that you can't just add to any client. (In contrast, you could always distribute a X509 certificate or use FORM, BASIC, or DIGEST login. But you can't add your client to a Windows Domain just in order to log in to an application.) Adding a fallback mechanism to SpnegoAuthenticator only may be much easier. What other solutions do you know? Carsten What other solutions do you know? - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Possible bug in http2 window size handling in tomcat 9.0.45
On 27/06/2021 12:05, Erik Nilsson wrote: We might have found an issue with the window size in http2 in Tomcat 9.0.45. Thanks for the heads up. 9.0.45 has fixes for all the known issues with window size management so this looks like a potential new bug. java.lang.IllegalStateException: Connection [483], Stream [21], Already waiting at org.apache.coyote.http2.WindowAllocationManager.waitFor(WindowAllocationManager.java:143) at org.apache.coyote.http2.WindowAllocationManager.waitForConnection(WindowAllocationManager.java:84) at org.apache.coyote.http2.Stream.waitForConnectionAllocation(Stream.java:257) at org.apache.coyote.http2.Http2UpgradeHandler.reserveWindowSize(Http2UpgradeHandler.java:903) at org.apache.coyote.http2.Stream$StreamOutputBuffer.flush(Stream.java:904) at org.apache.coyote.http2.Stream$StreamOutputBuffer.doWrite(Stream.java:823) at org.apache.coyote.http2.Http2OutputBuffer.doWrite(Http2OutputBuffer.java:59) at org.apache.coyote.Response.doWrite(Response.java:606) at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:340) at org.apache.catalina.connector.OutputBuffer.appendByteArray(OutputBuffer.java:753) at org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:682) at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:388) at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:366) at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:96) at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89) This problem occurs when we are using the nghttp client to receive data from our root page (nghttp -vnsay https://localhost). By decreasing the initial window size to 8 kb in nghttp (flag -w 13), the problem disappears. Another solution is to set the http2-connector attribute maxConcurrentStreamExecution="1". Hmm. That suggests some sort of concurrency issue. How easy is this to reproduce? I'll see if I can reproduce this locally but it may well depend on the structure of the page you are testing with. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: 500 instances of tomcat on the same server
On 26/06/2021 03:58, Eric Robinson wrote: We can run 75 to 125 instances of tomcat on a single Linux server with 12 cores and 128GB RAM. It works great. CPU is around 25%, our JVMs are not throwing OOMEs, iowait is minimal, and network traffic is about 30Mbps. We're happy with the results. Now we're upping the ante. We have a 48-core server with 1TB RAM, and we're planning to run 600+ tomcat instances on it simultaneously. What caveats or pitfalls should we watch out for? Are there any hard limits that would prevent this from working as expected? Nothing comes to mind from a Tomcat perspective. If there was going to be an issue, it would be with the OS or the hardware and you look to have thought all of that through. File limits are the only thing you didn't mention that I'd want to keep an eye on. Changing topic slightly, if there are changes we could make to Tomcat that would it easier to run and manage that many instances do let us know. We'd be happy to consider them. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Trouble with HTTP/2 during concurrent bulk data transfer (server -> client)
On 28/06/2021 06:14, Deshmukh, Kedar wrote: Any tentative time line when fix will be available in 9.0.X release ? Releases are typically made every month. The release usually happens some time in the second week of the month. The July releases are currently look like they will be comparatively early - possibly as soon as the end of this week. If you want to track release progress then I'd recommend following the dev@ list. Mark Thanks, Kedar -Original Message- From: Mark Thomas Sent: Friday, June 18, 2021 2:50 AM To: users@tomcat.apache.org Subject: Re: Trouble with HTTP/2 during concurrent bulk data transfer (server -> client) On 17/06/2021 09:26, Mark Thomas wrote: I think I might have found one contributing factor to this bug. I need to run a series of tests to determine whether I am seeing random variation in test results or a genuine effect. It was random effects but I believe I have now found the bug. Consider two threads, T1 and T2 writing HTTP/2 response bodies concurrently in the same HTTP/2 Connection. You'll need to have the code in front of you to follow what is going on The write: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Ftomcat%2Fblob%2Fmain%2Fjava%2Forg%2Fapache%2Ftomcat%2Futil%2Fnet%2FSocketWrapperBase.java%23L1364&data=04%7C01%7Cdkedar%40ptc.com%7C5df90eb802f84737230a08d931d5ce12%7Cb9921086ff774d0d828acb3381f678e2%7C0%7C0%7C637595616621142835%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=p2j7GZ6kDbeWc8%2BqnBGoadjhgV8w%2FcG8YnriPDeV%2F2g%3D&reserved=0 and the associated completion handler https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Ftomcat%2Fblob%2Fmain%2Fjava%2Forg%2Fapache%2Ftomcat%2Futil%2Fnet%2FSocketWrapperBase.java%23L1044&data=04%7C01%7Cdkedar%40ptc.com%7C5df90eb802f84737230a08d931d5ce12%7Cb9921086ff774d0d828acb3381f678e2%7C0%7C0%7C637595616621142835%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UVt7wgZ2GuKML2VUMh%2B58f7sK0cxdS0ZRAOs0gGiasQ%3D&reserved=0 The detail of the code is fairly complex but all you really need to keep in mind is the following: - the writePending semaphore ensures only one thread can write at a time - the state of the write is maintained in a OperationState instance that is stored in SocketWrapperBase.writeOperation (L1390) - the completion handler clears this state (L1050) and releases the semaphore (L1046) The sequence of events for a failure is as follows: - T1 obtains the write semaphore (L1366) - T1 creates an OperationState and sets writeOperation (L1390) - the async write for T1 completes and the completion handler is called - T1's completion handler releases the semaphore (L1046) - T2 obtains the write semaphore (L1366) - T2 creates an OperationState and sets writeOperation (L1390) - T1's completion handler clears writeOperation (L1050) - the async write for T2 does not complete and the socket is added to the Poller - The Poller signals the socket is ready for write - The Poller finds writeOperation is null so performs a normal dispatch for write - The async write times out as it never receives the notification from the Poller The fix is to swap the order of clearing writeOperation and releasing the semaphore. Concurrent reads will have the same problem and will be fixed by the same solution. Fix will be applied shortly. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
