Re: Test valve with tomcat-embed 9?
On 08/10/2021 11:43, Me Self wrote:
I would like to test a custom tomcat valve with tomcat-embed and junit. Is
that possible?
Found a few tomcat-embed samples on the web but most seem to only deal with
setting up a webapp - something along the lines:
@BeforeAll
public static void setup() throws LifecycleException {
Tomcat tomcat = new Tomcat();
tomcat.setPort(...);
StandardContext ctx = (StandardContext) tomcat.addWebapp("/", new
File("src/main/webapp/").getAbsolutePath());
What would I need to do to add a valve? And btw. it's a maven project so
the valve is compiled to "target/classes".
https://github.com/apache/tomcat/tree/main/test/org/apache/catalina/valves
Mark
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Test valve with tomcat-embed 9?
I would like to test a custom tomcat valve with tomcat-embed and junit. Is
that possible?
Found a few tomcat-embed samples on the web but most seem to only deal with
setting up a webapp - something along the lines:
@BeforeAll
public static void setup() throws LifecycleException {
Tomcat tomcat = new Tomcat();
tomcat.setPort(...);
StandardContext ctx = (StandardContext) tomcat.addWebapp("/", new
File("src/main/webapp/").getAbsolutePath());
What would I need to do to add a valve? And btw. it's a maven project so
the valve is compiled to "target/classes".
Re: Tomcat ant test failure
On 23/03/2020 12:43, Udupa, Ankitha K wrote: > Hi Mark, > > We have resolved similar errors like > "sun.security.validator.ValidatorException: PKIX path validation failed: > java.security.cert.CertPath > ValidatorException: timestamp check failed" > by downloading the certificates from latest tomcat source code > (https://github.com/apache/tomcat/tree/7.0.x/test/org/apache/tomcat/util/net). > > Please let me know if there is any other way to resolve this issue without > downloading the latest certificate once they are expired. Set the date on the server running the tests back far enough that the certificates appear to be valid? Mark > > Thanks and Regards, > Ankitha > > -Original Message- > From: Mark Thomas [mailto:ma...@apache.org] > Sent: Monday, March 23, 2020 3:08 PM > To: Tomcat Users List > Subject: Re: Tomcat ant test failure > > On 23/03/2020 07:18, Udupa, Ankitha K wrote: >> Hi, >> >> We are facing issue while triggering Ant Test with Tomcat 7.0.84 > > That version is over two years old and has a number of known security > vulnerabilities that are likely to apply to a typical installation. > > The Tomcat community typically provides support on a "first update to the > latest release of a supported version (7.0.103 in this instance), then > recreate then problem, then we'll try and help" basis. The further you are > away from the latest version of a supported release, the less likely you are > to receive help. > >> on HP UX IA/PA architecture with Java ( Java 7.0 in IA and Java 6.0 in PA). >> Please kindly help me to resolve these failures. > > Why? What are you trying to achieve by this? > > Each test will have a dedicated log file. You'll need to provide the log > files associated with each failed test if you want us to try and help. > > Best guess is that the async state errors are highlighting edge case bugs > that are fixed in a later release and the TLS bugs are probably due to > differences in the HP JVMs. No idea about the Comet errors. > > Mark > > >> TEST-org.apache.catalina.comet.TestCometProcessor.NIO.txt:ERROR:null >> TEST-org.apache.catalina.comet.TestCometProcessor.NIO.txt:ERROR:null >> TEST-org.apache.catalina.connector.TestCoyoteAdapter.BIO.txt:Exception >> in thread "testBug54928" java.lang.IllegalStateException: Calling >> [asyncComplete()] is not valid for a request with Async state >> [MUST_ERROR] >> TEST-org.apache.catalina.connector.TestCoyoteAdapter.NIO.txt:Exception >> in thread "testBug54928" java.lang.IllegalStateException: Calling >> [asyncComplete()] is not valid for a request with Async state >> [MUST_ERROR] >> TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt: Caused an >> ERROR >> TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt: Caused an ERROR >> TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Caused an ERROR >> TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Caused an ERROR >> TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Caused an ERROR >> TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Caused an ERROR >> TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt: Caused an ERROR >> TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Caused an ERROR >> TEST-org.apache.tomcat.websocket.TestWebSocketFrameClientSSL.BIO.txt: >> Caused an ERROR >> TEST-org.apache.tomcat.websocket.
RE: Tomcat ant test failure
Hi Mark, We have resolved similar errors like "sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPath ValidatorException: timestamp check failed" by downloading the certificates from latest tomcat source code (https://github.com/apache/tomcat/tree/7.0.x/test/org/apache/tomcat/util/net). Please let me know if there is any other way to resolve this issue without downloading the latest certificate once they are expired. Thanks and Regards, Ankitha -Original Message- From: Mark Thomas [mailto:ma...@apache.org] Sent: Monday, March 23, 2020 3:08 PM To: Tomcat Users List Subject: Re: Tomcat ant test failure On 23/03/2020 07:18, Udupa, Ankitha K wrote: > Hi, > > We are facing issue while triggering Ant Test with Tomcat 7.0.84 That version is over two years old and has a number of known security vulnerabilities that are likely to apply to a typical installation. The Tomcat community typically provides support on a "first update to the latest release of a supported version (7.0.103 in this instance), then recreate then problem, then we'll try and help" basis. The further you are away from the latest version of a supported release, the less likely you are to receive help. > on HP UX IA/PA architecture with Java ( Java 7.0 in IA and Java 6.0 in PA). > Please kindly help me to resolve these failures. Why? What are you trying to achieve by this? Each test will have a dedicated log file. You'll need to provide the log files associated with each failed test if you want us to try and help. Best guess is that the async state errors are highlighting edge case bugs that are fixed in a later release and the TLS bugs are probably due to differences in the HP JVMs. No idea about the Comet errors. Mark > TEST-org.apache.catalina.comet.TestCometProcessor.NIO.txt:ERROR:null > TEST-org.apache.catalina.comet.TestCometProcessor.NIO.txt:ERROR:null > TEST-org.apache.catalina.connector.TestCoyoteAdapter.BIO.txt:Exception > in thread "testBug54928" java.lang.IllegalStateException: Calling > [asyncComplete()] is not valid for a request with Async state > [MUST_ERROR] > TEST-org.apache.catalina.connector.TestCoyoteAdapter.NIO.txt:Exception > in thread "testBug54928" java.lang.IllegalStateException: Calling > [asyncComplete()] is not valid for a request with Async state > [MUST_ERROR] > TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt: Caused an > ERROR > TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Caused an ERROR > TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Caused an ERROR > TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Caused an ERROR > TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Caused an ERROR > TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Caused an ERROR > TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Caused an ERROR > TEST-org.apache.tomcat.websocket.TestWebSocketFrameClientSSL.BIO.txt: > Caused an ERROR > TEST-org.apache.tomcat.websocket.TestWebSocketFrameClientSSL.NIO.txt: > Caused an ERROR > TEST-org.apache.tomcat.websocket.TestWebSocketFrameClientSSL.NIO.txt: > Caused an ERROR > TEST-org.apache.tomcat.websocket.TestWsWebSocketContainer.BIO.txt: > Caused an ERROR > TEST-org.apache.tomcat.websocket.TestWsWebSocketContainer.NIO.txt: > Caused an ERROR > > Thanks and Regards, > Ankitha > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat ant test failure
On 23/03/2020 07:18, Udupa, Ankitha K wrote: > Hi, > > We are facing issue while triggering Ant Test with Tomcat 7.0.84 That version is over two years old and has a number of known security vulnerabilities that are likely to apply to a typical installation. The Tomcat community typically provides support on a "first update to the latest release of a supported version (7.0.103 in this instance), then recreate then problem, then we'll try and help" basis. The further you are away from the latest version of a supported release, the less likely you are to receive help. > on HP UX IA/PA architecture with Java ( Java 7.0 in IA and Java 6.0 in PA). > Please kindly help me to resolve these failures. Why? What are you trying to achieve by this? Each test will have a dedicated log file. You'll need to provide the log files associated with each failed test if you want us to try and help. Best guess is that the async state errors are highlighting edge case bugs that are fixed in a later release and the TLS bugs are probably due to differences in the HP JVMs. No idea about the Comet errors. Mark > TEST-org.apache.catalina.comet.TestCometProcessor.NIO.txt:ERROR:null > TEST-org.apache.catalina.comet.TestCometProcessor.NIO.txt:ERROR:null > TEST-org.apache.catalina.connector.TestCoyoteAdapter.BIO.txt:Exception in > thread "testBug54928" java.lang.IllegalStateException: Calling > [asyncComplete()] is not valid for a request with Async state [MUST_ERROR] > TEST-org.apache.catalina.connector.TestCoyoteAdapter.NIO.txt:Exception in > thread "testBug54928" java.lang.IllegalStateException: Calling > [asyncComplete()] is not valid for a request with Async state [MUST_ERROR] > TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt: Caused an ERROR > TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Caused an ERROR > TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Caused an ERROR > TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Caused an ERROR > TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Caused an ERROR > TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Caused an ERROR > TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Caused an ERROR > TEST-org.apache.tomcat.websocket.TestWebSocketFrameClientSSL.BIO.txt: > Caused an ERROR > TEST-org.apache.tomcat.websocket.TestWebSocketFrameClientSSL.NIO.txt: > Caused an ERROR > TEST-org.apache.tomcat.websocket.TestWebSocketFrameClientSSL.NIO.txt: > Caused an ERROR > TEST-org.apache.tomcat.websocket.TestWsWebSocketContainer.BIO.txt: > Caused an ERROR > TEST-org.apache.tomcat.websocket.TestWsWebSocketContainer.NIO.txt: > Caused an ERROR > > Thanks and Regards, > Ankitha > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Tomcat ant test failure
Hi, We are facing issue while triggering Ant Test with Tomcat 7.0.84 on HP UX IA/PA architecture with Java ( Java 7.0 in IA and Java 6.0 in PA). Please kindly help me to resolve these failures. TEST-org.apache.catalina.comet.TestCometProcessor.NIO.txt:ERROR:null TEST-org.apache.catalina.comet.TestCometProcessor.NIO.txt:ERROR:null TEST-org.apache.catalina.connector.TestCoyoteAdapter.BIO.txt:Exception in thread "testBug54928" java.lang.IllegalStateException: Calling [asyncComplete()] is not valid for a request with Async state [MUST_ERROR] TEST-org.apache.catalina.connector.TestCoyoteAdapter.NIO.txt:Exception in thread "testBug54928" java.lang.IllegalStateException: Calling [asyncComplete()] is not valid for a request with Async state [MUST_ERROR] TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt: Caused an ERROR TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Caused an ERROR TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Caused an ERROR TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Caused an ERROR TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Caused an ERROR TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Caused an ERROR TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Caused an ERROR TEST-org.apache.tomcat.websocket.TestWebSocketFrameClientSSL.BIO.txt: Caused an ERROR TEST-org.apache.tomcat.websocket.TestWebSocketFrameClientSSL.NIO.txt: Caused an ERROR TEST-org.apache.tomcat.websocket.TestWebSocketFrameClientSSL.NIO.txt: Caused an ERROR TEST-org.apache.tomcat.websocket.TestWsWebSocketContainer.BIO.txt: Caused an ERROR TEST-org.apache.tomcat.websocket.TestWsWebSocketContainer.NIO.txt: Caused an ERROR Thanks and Regards, Ankitha
RE: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
Hi, -Original Message- From: Mark Thomas Sent: Thursday, October 10, 2019 3:54 AM To: users@tomcat.apache.org Subject: Re: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib On 09/10/2019 22:58, Rhuberg,Anthony wrote: > StandardRoot.gc() unconditionally closes the web application jars in Tomcat > 9... every 10s by default or configurable by changing > backgroundProcessorDelay. > > So then the problem is Our web application is calling > class.getResourceAsStream() to read some static files... and this is causing > the jars to be read again? It sounds like it. > This was not a problem with Tomcat 7 because the jars were not closed if they > were read within the jarOpenInterval setting? Or that the frequency that the JARs were closed was low enough that the performance impact was not noticed. Mark > > Thanks, > Tony > > -Original Message- > From: Mark Thomas > Sent: Wednesday, October 09, 2019 5:32 PM > To: users@tomcat.apache.org > Subject: Re: Performance test with Tomcat 9 shows increased cpu/disk > usage because of repeated opening/closing of jars in WEB-INF/lib > > On 09/10/2019 21:03, Rhuberg,Anthony wrote: >> This seems to alleviate the issue... in context.xml (sc-test#sc.xml) >> > swallowOutput="true" backgroundProcessorDelay="90"> >> >> Not sure if this is the context reload trigger... i.e. the >> webappLoader.backgroundProcess method is triggered every 90 seconds... > > It isn't. It is StandardRoot.gc() that is unloading the cached JAR entries. > > Mark > > - Here is an update as to the causes of the repeated reloads our web application jars. Some of the following may be "bad" practices (at least not optimal), but seem to be commonplace. In other cases, the behavior was unexpected and may warrant a change to WebResourceRoot to allow the jars to remain open or otherwise control that behavior. Have 2 questions: 1. Could the WebResourceRoot be changed to provide a way to control when the jars are closed? 2. Any concerns with any of the solutions below? Has anyone used these in the past or currently and found any problems? 1. JAXP related factories We frequently use DocumentBuilderFactory.newInstance() to obtain an instance of the factory implementation and others like TransformerFactory, XPathFactory, DatatypeFactory, and SAXParserFactory. This will trigger a jar reload when the service loader attempts to find the implementation class. We decided to set the applicable system properties to avoid the search and thus eliminate this reason for the jar reload. See below for the factories configured; this change reduced JVM CPU usage percentage in a load test by 26% (using Tomcat 7, so expect similar in Tomcat 9, plus the elimination of the jar reload). 2. SOAP web service endpoints Each time we invoke a SOAP web service we are creating an instance of javax.xml.ws.Service and a client proxy that implements the applicable web service contract (Service.getPort). When these objects are created the service loader is used to find the implementation for the web service client. We are using the Apache CXF client (cxf-rt-frontend-jaxws/META-INF/services/javax.xml.ws.spi.Provider). In addition, the JAXB bindings are created. These actions occur every time we invoke a SOAP web service; this subsequently searches for the implementation classes triggering the web application jars to be reloaded. We decided to create the service end-points once and reuse. Creating the objects once and reusing avoids the cost of repeatedly creating the instances. The CXF instances are thread safe (https://cxf.apache.org/faq.html#FAQ-AreJAX-WSclientproxiesthreadsafe?). Testing to verify the objects are thread safe is in progress. 3. When we invoke a REST web service, we create an instance of the Spring RestTemplate. The template then registers various mappers; the MappingJackson2HttpMessageConverter uses Jackson2ObjectMapperBuilder.registerWellKnownModulesIfAvailable() to register a few optional modules. Each time the template is created a classpath search is performed that can trigger the jars to be reloaded. See: https://github.com/spring-projects/spring-framework/search?utf8=%E2%9C%93=registerWellKnownModulesIfAvailable=. The optional modules include: Jdk7Module, Jdk8Module, and JavaModule. Adding these jars allows the classes to be found and then cached, thus preventing subsequent jar reloads: jackson-datatype-jsr310-2.10.1.jar jackson-datatype-jdk7-2.6.7.jar, and jackson-datatype-jdk8-2.10.1.jar. 4. When transforming XML documents, we repeatedly read the .xsl file using getResourceAsStream(sometransform.xsl). This is cached by default for 5 seconds. We increased the
RE: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
We continue to profile our application and have found instances of using TransformerFactory.newInstance(). The more concerning problem is third party dependencies that do this or something similar we could not fix. Jar modified times are not changing - I did not mean to state or otherwise imply that. -Original Message- From: john.e.gr...@wellsfargo.com.INVALID Sent: Thursday, October 10, 2019 6:54 PM To: users@tomcat.apache.org Subject: RE: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib Tony, > -Original Message- > From: Rhuberg,Anthony > Sent: Thursday, October 10, 2019 5:22 PM > To: Tomcat Users List > Subject: RE: Performance test with Tomcat 9 shows increased cpu/disk > usage because of repeated opening/closing of jars in WEB-INF/lib > > We are still investigating what specific classloader reads that would > trigger the repeated reload of the web-inf/lib/*.jars. > > One example we found is the use of > javax.xml.transform.TransformerFactory.newInstance(). One of its > features is to determine the implementation by searching for the > configuration specified by > META-INF/services/javax.xml.transform.TransformerFactory in jars available to > the runtime. > Yep. I was wondering about that. I have no idea why this behavior would be different in Tomcat 9, though. Did you change Java versions also? Is the classpath longer? I also have no idea about the jar modification date changes that you're seeing. I do have first-hand experience with the performance of TransformerFactory.newInstance(). In general, anything that uses the service locator under the covers, such as TransformerFactory, DocumentBuilderFactory, etc, should be reused if possible. Those XML factories are thread safe, though the things created by the factories are typically not thread safe. You should create it once, configure it once, and use it as many times as necessary. For example, you would create one TransformerFactory, then call newTransformer() as many times as necessary. John B CB [ X ܚX KK[XZ[ \ \ ][ X ܚX P X ] \X K ܙ B ܈Y][ۘ[ [X[ K[XZ[ \ \ Z[ X ] \X K ܙ B CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
Tony, > -Original Message- > From: Rhuberg,Anthony > Sent: Thursday, October 10, 2019 5:22 PM > To: Tomcat Users List > Subject: RE: Performance test with Tomcat 9 shows increased cpu/disk usage > because of repeated opening/closing of jars in WEB-INF/lib > > We are still investigating what specific classloader reads that would trigger > the repeated reload of the web-inf/lib/*.jars. > > One example we found is the use of > javax.xml.transform.TransformerFactory.newInstance(). One of its features > is to determine the implementation by searching for the configuration > specified by META-INF/services/javax.xml.transform.TransformerFactory in > jars available to the runtime. > Yep. I was wondering about that. I have no idea why this behavior would be different in Tomcat 9, though. Did you change Java versions also? Is the classpath longer? I also have no idea about the jar modification date changes that you're seeing. I do have first-hand experience with the performance of TransformerFactory.newInstance(). In general, anything that uses the service locator under the covers, such as TransformerFactory, DocumentBuilderFactory, etc, should be reused if possible. Those XML factories are thread safe, though the things created by the factories are typically not thread safe. You should create it once, configure it once, and use it as many times as necessary. For example, you would create one TransformerFactory, then call newTransformer() as many times as necessary. John
RE: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
We are still investigating what specific classloader reads that would trigger the repeated reload of the web-inf/lib/*.jars. One example we found is the use of javax.xml.transform.TransformerFactory.newInstance(). One of its features is to determine the implementation by searching for the configuration specified by META-INF/services/javax.xml.transform.TransformerFactory in jars available to the runtime. It seems users of Tomcat 9 would need to understand the management of the jar resource handles when using the TransformerFactory class - assuming it is triggering the reload of all the related application jars - which if happening enough can cause the high cpu/disk usage as we observed in our performance tests. Thanks -Original Message- From: Mark Thomas Sent: Thursday, October 10, 2019 3:54 AM To: users@tomcat.apache.org Subject: Re: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib On 09/10/2019 22:58, Rhuberg,Anthony wrote: > StandardRoot.gc() unconditionally closes the web application jars in Tomcat > 9... every 10s by default or configurable by changing > backgroundProcessorDelay. > > So then the problem is Our web application is calling > class.getResourceAsStream() to read some static files... and this is causing > the jars to be read again? It sounds like it. > This was not a problem with Tomcat 7 because the jars were not closed if they > were read within the jarOpenInterval setting? Or that the frequency that the JARs were closed was low enough that the performance impact was not noticed. Mark > > Thanks, > Tony > > -Original Message- > From: Mark Thomas > Sent: Wednesday, October 09, 2019 5:32 PM > To: users@tomcat.apache.org > Subject: Re: Performance test with Tomcat 9 shows increased cpu/disk > usage because of repeated opening/closing of jars in WEB-INF/lib > > On 09/10/2019 21:03, Rhuberg,Anthony wrote: >> This seems to alleviate the issue... in context.xml (sc-test#sc.xml) >> > swallowOutput="true" backgroundProcessorDelay="90"> >> >> Not sure if this is the context reload trigger... i.e. the >> webappLoader.backgroundProcess method is triggered every 90 seconds... > > It isn't. It is StandardRoot.gc() that is unloading the cached JAR entries. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > CONFIDENTIALITY NOTICE This message and any included attachments are from > Cerner Corporation and are intended only for the addressee. The information > contained in this message is confidential and may constitute inside or > non-public information under international, federal, or state securities > laws. Unauthorized forwarding, printing, copying, distribution, or use of > such information is strictly prohibited and may be unlawful. If you are not > the addressee, please promptly delete this message and notify the sender of > the delivery error by e-mail or you may call Cerner's corporate offices in > Kansas City, Missouri, U.S.A at (+1) (816)221-1024. > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
On 09/10/2019 22:58, Rhuberg,Anthony wrote: > StandardRoot.gc() unconditionally closes the web application jars in Tomcat > 9... every 10s by default or configurable by changing > backgroundProcessorDelay. > > So then the problem is Our web application is calling > class.getResourceAsStream() to read some static files... and this is causing > the jars to be read again? It sounds like it. > This was not a problem with Tomcat 7 because the jars were not closed if they > were read within the jarOpenInterval setting? Or that the frequency that the JARs were closed was low enough that the performance impact was not noticed. Mark > > Thanks, > Tony > > -Original Message- > From: Mark Thomas > Sent: Wednesday, October 09, 2019 5:32 PM > To: users@tomcat.apache.org > Subject: Re: Performance test with Tomcat 9 shows increased cpu/disk usage > because of repeated opening/closing of jars in WEB-INF/lib > > On 09/10/2019 21:03, Rhuberg,Anthony wrote: >> This seems to alleviate the issue... in context.xml (sc-test#sc.xml) >> > swallowOutput="true" backgroundProcessorDelay="90"> >> >> Not sure if this is the context reload trigger... i.e. the >> webappLoader.backgroundProcess method is triggered every 90 seconds... > > It isn't. It is StandardRoot.gc() that is unloading the cached JAR entries. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > CONFIDENTIALITY NOTICE This message and any included attachments are from > Cerner Corporation and are intended only for the addressee. The information > contained in this message is confidential and may constitute inside or > non-public information under international, federal, or state securities > laws. Unauthorized forwarding, printing, copying, distribution, or use of > such information is strictly prohibited and may be unlawful. If you are not > the addressee, please promptly delete this message and notify the sender of > the delivery error by e-mail or you may call Cerner's corporate offices in > Kansas City, Missouri, U.S.A at (+1) (816)221-1024. > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
I have not seen the trace message for modified()... So my understanding of this is wrong. -Original Message- From: Paul Carter-Brown Sent: Wednesday, October 09, 2019 4:42 PM To: Tomcat Users List Subject: Re: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib Hi Anthony Have you turned debug logging on to see what it is picking up as modified? On Wed, 09 Oct 2019, 22:24 Rhuberg,Anthony, wrote: > Thanks for your responses. > > I understand that re-reading the static files is not optimal, but how > does reading files from the jar affect them being reloaded... In > Tomcat 7, that seemed to prevent the jars from being reloaded since > the accessTime was changed; in Tomcat 8, it seems the modified() > method uses the file modified time - not sure what we are doing to > cause the modified time to change - the files on the filesystem are > not changing - unless there is something happening in the classes folder. > > Thanks > > -Original Message- > From: Mark Thomas > Sent: Wednesday, October 09, 2019 3:56 PM > To: users@tomcat.apache.org > Subject: Re: Performance test with Tomcat 9 shows increased cpu/disk > usage because of repeated opening/closing of jars in WEB-INF/lib > > On 09/10/2019 20:08, Rhuberg,Anthony wrote: > > On the other thread: Is this genuine class loading (in which case > > the response below is correct) or is the application reading the > > .class > files as if they were resources (in which case a different answer applies)? > > Mostly, we are talking about the default class loader; however, the > application does read resources from the jar files - mostly static > files that are stored within the package structure are read (for > example: log4j, property files, other static content). Would this > affect the reload of the jar files? > > If an application was reading the same resource from a JAR over and > over again, that may have a performance impact. Increasing the TTL for > cached resources would probably address that although, arguably, the > better solution in that case would be to fix the app to load it once and > cache it. > > > -Original Message- > > From: Rhuberg,Anthony > > Sent: Wednesday, October 09, 2019 2:53 PM > > To: Tomcat Users List > > Subject: RE: Performance test with Tomcat 9 shows increased cpu/disk > > usage because of repeated opening/closing of jars in WEB-INF/lib > > > > Just noticed another thread on this topic: RE: Tomcat discards and > reloads the jar files from the webapps folder. > > > > Setting the Engine attribute backgroundProcessorDelay to 90s does > > reduce > the open/close cycle of the jars. > > Thanks for the confirmation. > > > The other thread also mentioned a potential configuration to keep > > the > jars open longer by default. This could be helpful - not sure we need > to close them at all. > > If the JARs aren't closed it can cause file locking issues. We could > include a disable option if we did make the Jar unload frequency > configurable. > > > Alternatively, would there be any gain to 'explode' the jars into > > class > files on the filesystem? > > Maybe. Best bet it to try it for your app and see. > > Mark > > > > -Original Message- > > From: Rhuberg,Anthony > > Sent: Wednesday, October 09, 2019 2:17 PM > > To: users@tomcat.apache.org > > Subject: Performance test with Tomcat 9 shows increased cpu/disk > > usage because of repeated opening/closing of jars in WEB-INF/lib > > > > Background: > > In the last few months we migrated our web application from Tomcat > 7.0.55 to Tomcat 9.0.19 (26). That transition was relatively > straightforward until we reviewed the results of our performance tests. > Those tests showed an increase in CPU usage and disk I/O on our > Windows > 2012 server. When we switch back to Tomcat 7, the metrics for CPU/disk > usage were significantly lower (similar to previous tests). > > > > As we investigated the cause of the increase in cpu/disk usage we > > found > the tomcat process was repeatedly opening and closing the jars of our > web application. The process profile showed all the jars in > WEB-INF/lib being closed and reopened within seconds while the > application was processing requests (and to some extent when quite). > Now we are trying to determine why. > > > > We have a general understanding that the WebappClassLoaderBase.java > implementation changed in Tomcat 8.5. The Tomcat 7 implementation > provided for the configuration of jarOpenInterval (which we used the > default = 90s). The latest implementati
RE: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
StandardRoot.gc() unconditionally closes the web application jars in Tomcat 9... every 10s by default or configurable by changing backgroundProcessorDelay. So then the problem is Our web application is calling class.getResourceAsStream() to read some static files... and this is causing the jars to be read again? This was not a problem with Tomcat 7 because the jars were not closed if they were read within the jarOpenInterval setting? Thanks, Tony -Original Message- From: Mark Thomas Sent: Wednesday, October 09, 2019 5:32 PM To: users@tomcat.apache.org Subject: Re: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib On 09/10/2019 21:03, Rhuberg,Anthony wrote: > This seems to alleviate the issue... in context.xml (sc-test#sc.xml) > swallowOutput="true" backgroundProcessorDelay="90"> > > Not sure if this is the context reload trigger... i.e. the > webappLoader.backgroundProcess method is triggered every 90 seconds... It isn't. It is StandardRoot.gc() that is unloading the cached JAR entries. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
Re: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
On 09/10/2019 21:03, Rhuberg,Anthony wrote: > This seems to alleviate the issue... in context.xml (sc-test#sc.xml) > backgroundProcessorDelay="90"> > > Not sure if this is the context reload trigger... i.e. the > webappLoader.backgroundProcess method is triggered every 90 seconds... It isn't. It is StandardRoot.gc() that is unloading the cached JAR entries. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
Hi Anthony Have you turned debug logging on to see what it is picking up as modified? On Wed, 09 Oct 2019, 22:24 Rhuberg,Anthony, wrote: > Thanks for your responses. > > I understand that re-reading the static files is not optimal, but how does > reading files from the jar affect them being reloaded... In Tomcat 7, that > seemed to prevent the jars from being reloaded since the accessTime was > changed; in Tomcat 8, it seems the modified() method uses the file modified > time - not sure what we are doing to cause the modified time to change - > the files on the filesystem are not changing - unless there is something > happening in the classes folder. > > Thanks > > -Original Message- > From: Mark Thomas > Sent: Wednesday, October 09, 2019 3:56 PM > To: users@tomcat.apache.org > Subject: Re: Performance test with Tomcat 9 shows increased cpu/disk usage > because of repeated opening/closing of jars in WEB-INF/lib > > On 09/10/2019 20:08, Rhuberg,Anthony wrote: > > On the other thread: Is this genuine class loading (in which case the > > response below is correct) or is the application reading the .class > files as if they were resources (in which case a different answer applies)? > > Mostly, we are talking about the default class loader; however, the > application does read resources from the jar files - mostly static files > that are stored within the package structure are read (for example: log4j, > property files, other static content). Would this affect the reload of the > jar files? > > If an application was reading the same resource from a JAR over and over > again, that may have a performance impact. Increasing the TTL for cached > resources would probably address that although, arguably, the better > solution in that case would be to fix the app to load it once and cache it. > > > -Original Message- > > From: Rhuberg,Anthony > > Sent: Wednesday, October 09, 2019 2:53 PM > > To: Tomcat Users List > > Subject: RE: Performance test with Tomcat 9 shows increased cpu/disk > > usage because of repeated opening/closing of jars in WEB-INF/lib > > > > Just noticed another thread on this topic: RE: Tomcat discards and > reloads the jar files from the webapps folder. > > > > Setting the Engine attribute backgroundProcessorDelay to 90s does reduce > the open/close cycle of the jars. > > Thanks for the confirmation. > > > The other thread also mentioned a potential configuration to keep the > jars open longer by default. This could be helpful - not sure we need to > close them at all. > > If the JARs aren't closed it can cause file locking issues. We could > include a disable option if we did make the Jar unload frequency > configurable. > > > Alternatively, would there be any gain to 'explode' the jars into class > files on the filesystem? > > Maybe. Best bet it to try it for your app and see. > > Mark > > > > -Original Message- > > From: Rhuberg,Anthony > > Sent: Wednesday, October 09, 2019 2:17 PM > > To: users@tomcat.apache.org > > Subject: Performance test with Tomcat 9 shows increased cpu/disk usage > > because of repeated opening/closing of jars in WEB-INF/lib > > > > Background: > > In the last few months we migrated our web application from Tomcat > 7.0.55 to Tomcat 9.0.19 (26). That transition was relatively > straightforward until we reviewed the results of our performance tests. > Those tests showed an increase in CPU usage and disk I/O on our Windows > 2012 server. When we switch back to Tomcat 7, the metrics for CPU/disk > usage were significantly lower (similar to previous tests). > > > > As we investigated the cause of the increase in cpu/disk usage we found > the tomcat process was repeatedly opening and closing the jars of our web > application. The process profile showed all the jars in WEB-INF/lib being > closed and reopened within seconds while the application was processing > requests (and to some extent when quite). Now we are trying to determine > why. > > > > We have a general understanding that the WebappClassLoaderBase.java > implementation changed in Tomcat 8.5. The Tomcat 7 implementation provided > for the configuration of jarOpenInterval (which we used the default = > 90s). The latest implementation appears to track the jar modification times > with a hash map called jarModificationTimes, but we are unable to find a > similar time-to-live configuration or what triggers the lifecycle listener > to close the jars (what calls WebappClassLoaderBase.modified() or > destroy()) - assuming we understand this lifecycle behavior. > > > > The server is configured mostly with defa
RE: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
Thanks for your responses. I understand that re-reading the static files is not optimal, but how does reading files from the jar affect them being reloaded... In Tomcat 7, that seemed to prevent the jars from being reloaded since the accessTime was changed; in Tomcat 8, it seems the modified() method uses the file modified time - not sure what we are doing to cause the modified time to change - the files on the filesystem are not changing - unless there is something happening in the classes folder. Thanks -Original Message- From: Mark Thomas Sent: Wednesday, October 09, 2019 3:56 PM To: users@tomcat.apache.org Subject: Re: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib On 09/10/2019 20:08, Rhuberg,Anthony wrote: > On the other thread: Is this genuine class loading (in which case the > response below is correct) or is the application reading the .class files as > if they were resources (in which case a different answer applies)? > Mostly, we are talking about the default class loader; however, the > application does read resources from the jar files - mostly static files that > are stored within the package structure are read (for example: log4j, > property files, other static content). Would this affect the reload of the > jar files? If an application was reading the same resource from a JAR over and over again, that may have a performance impact. Increasing the TTL for cached resources would probably address that although, arguably, the better solution in that case would be to fix the app to load it once and cache it. > -Original Message- > From: Rhuberg,Anthony > Sent: Wednesday, October 09, 2019 2:53 PM > To: Tomcat Users List > Subject: RE: Performance test with Tomcat 9 shows increased cpu/disk > usage because of repeated opening/closing of jars in WEB-INF/lib > > Just noticed another thread on this topic: RE: Tomcat discards and reloads > the jar files from the webapps folder. > > Setting the Engine attribute backgroundProcessorDelay to 90s does reduce the > open/close cycle of the jars. Thanks for the confirmation. > The other thread also mentioned a potential configuration to keep the jars > open longer by default. This could be helpful - not sure we need to close > them at all. If the JARs aren't closed it can cause file locking issues. We could include a disable option if we did make the Jar unload frequency configurable. > Alternatively, would there be any gain to 'explode' the jars into class files > on the filesystem? Maybe. Best bet it to try it for your app and see. Mark > -Original Message- > From: Rhuberg,Anthony > Sent: Wednesday, October 09, 2019 2:17 PM > To: users@tomcat.apache.org > Subject: Performance test with Tomcat 9 shows increased cpu/disk usage > because of repeated opening/closing of jars in WEB-INF/lib > > Background: > In the last few months we migrated our web application from Tomcat 7.0.55 to > Tomcat 9.0.19 (26). That transition was relatively straightforward until we > reviewed the results of our performance tests. Those tests showed an increase > in CPU usage and disk I/O on our Windows 2012 server. When we switch back to > Tomcat 7, the metrics for CPU/disk usage were significantly lower (similar to > previous tests). > > As we investigated the cause of the increase in cpu/disk usage we found the > tomcat process was repeatedly opening and closing the jars of our web > application. The process profile showed all the jars in WEB-INF/lib being > closed and reopened within seconds while the application was processing > requests (and to some extent when quite). Now we are trying to determine why. > > We have a general understanding that the WebappClassLoaderBase.java > implementation changed in Tomcat 8.5. The Tomcat 7 implementation provided > for the configuration of jarOpenInterval (which we used the default = 90s). > The latest implementation appears to track the jar modification times with a > hash map called jarModificationTimes, but we are unable to find a similar > time-to-live configuration or what triggers the lifecycle listener to close > the jars (what calls WebappClassLoaderBase.modified() or destroy()) - > assuming we understand this lifecycle behavior. > > The server is configured mostly with default settings. The Context reloadable > property is NOT set. We have tried increasing the cache time of the Resources > cacheTtl property as the default of 5s seemed close to the time the jars were > reloaded. We have not been successful. > > Question: > What is the cause of the repeated reloading of our web application jars? What > can trigger the class loader to close all the file handles related to the
RE: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
This seems to alleviate the issue... in context.xml (sc-test#sc.xml)
Not sure if this is the context reload trigger... i.e. the
webappLoader.backgroundProcess method is triggered every 90 seconds... the
reloadable property on Context is NOT set (default = false). So then how does
modified() evaluate to true?
The
https://github.com/apache/tomcat/blob/65aec2ac7fb4980694291e66efd360195dfa8d73/java/org/apache/catalina/loader/WebappLoader.java...
/**
* Execute a periodic task, such as reloading, etc. This method will be
* invoked inside the classloading context of this container. Unexpected
* throwables will be caught and logged.
*/
@Override
public void backgroundProcess() {
if (reloadable && modified()) {
try {
Thread.currentThread().setContextClassLoader
(WebappLoader.class.getClassLoader());
if (context != null) {
context.reload();
}
} finally {
if (context != null && context.getLoader() != null) {
Thread.currentThread().setContextClassLoader
(context.getLoader().getClassLoader());
}
}
}
}
I do not understand how modified = true. Why would the jar date/time be changed?
https://github.com/apache/tomcat/blob/8.5.x/java/org/apache/catalina/loader/WebappClassLoaderBase.java
/**
* Have one or more classes or resources been modified so that a reload
* is appropriate?
* @return true if there's been a modification
*/
public boolean modified() {
if (log.isDebugEnabled())
log.debug("modified()");
for (Entry entry : resourceEntries.entrySet()) {
long cachedLastModified = entry.getValue().lastModified;
long lastModified = resources.getClassLoaderResource(
entry.getKey()).getLastModified();
if (lastModified != cachedLastModified) {
if( log.isDebugEnabled() )
log.debug(sm.getString("webappClassLoader.resourceModified",
entry.getKey(),
new Date(cachedLastModified),
new Date(lastModified)));
return true;
}
}
// Check if JARs have been added or removed
WebResource[] jars = resources.listResources("/WEB-INF/lib");
// Filter out non-JAR resources
int jarCount = 0;
for (WebResource jar : jars) {
if (jar.getName().endsWith(".jar") && jar.isFile() &&
jar.canRead()) {
jarCount++;
Long recordedLastModified =
jarModificationTimes.get(jar.getName());
if (recordedLastModified == null) {
// Jar has been added
log.info(sm.getString("webappClassLoader.jarsAdded",
resources.getContext().getName()));
return true;
}
if (recordedLastModified.longValue() != jar.getLastModified()) {
// Jar has been changed
log.info(sm.getString("webappClassLoader.jarsModified",
resources.getContext().getName()));
return true;
}
}
}
if (jarCount < jarModificationTimes.size()){
log.info(sm.getString("webappClassLoader.jarsRemoved",
resources.getContext().getName()));
return true;
}
// No classes have been modified
return false;
}
-Original Message-
From: Rhuberg,Anthony
Sent: Wednesday, October 09, 2019 3:08 PM
To: Tomcat Users List
Subject: RE: Performance test with Tomcat 9 shows increased cpu/disk usage
because of repeated opening/closing of jars in WEB-INF/lib
On the other thread: Is this genuine class loading (in which case the response
below is correct) or is the application reading the .class files as if they
were resources (in which case a different answer applies)?
Mostly, we are talking about the default class loader; however, the application
does read resources from the jar files - mostly static files that are stored
within the package structure are read (for example: log4j, property files,
other static content). Would this affect the reload of the jar files?
-Original Message-
From: Rhuberg,Anthony
Sent: Wednesday, October 09, 2019 2:53 PM
To: Tomcat Users List
Subject: RE: Performance test with Tomcat 9 shows increased cpu/disk usage
because of repeated opening/closing of jars in WEB-INF/lib
Just noticed another thread on this topic: RE: Tomcat discards and reloads the
jar files from the webapps folder.
Setting the Engine attribute backgroundProcessorDelay to 90s does red
Re: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
On 09/10/2019 20:08, Rhuberg,Anthony wrote: > On the other thread: Is this genuine class loading (in which case the > response below is correct) or is the application reading the .class files as > if they were > resources (in which case a different answer applies)? > Mostly, we are talking about the default class loader; however, the > application does read resources from the jar files - mostly static files that > are stored within the package structure are read (for example: log4j, > property files, other static content). Would this affect the reload of the > jar files? If an application was reading the same resource from a JAR over and over again, that may have a performance impact. Increasing the TTL for cached resources would probably address that although, arguably, the better solution in that case would be to fix the app to load it once and cache it. > -Original Message- > From: Rhuberg,Anthony > Sent: Wednesday, October 09, 2019 2:53 PM > To: Tomcat Users List > Subject: RE: Performance test with Tomcat 9 shows increased cpu/disk usage > because of repeated opening/closing of jars in WEB-INF/lib > > Just noticed another thread on this topic: RE: Tomcat discards and reloads > the jar files from the webapps folder. > > Setting the Engine attribute backgroundProcessorDelay to 90s does reduce the > open/close cycle of the jars. Thanks for the confirmation. > The other thread also mentioned a potential configuration to keep the jars > open longer by default. This could be helpful - not sure we need to close > them at all. If the JARs aren't closed it can cause file locking issues. We could include a disable option if we did make the Jar unload frequency configurable. > Alternatively, would there be any gain to 'explode' the jars into class files > on the filesystem? Maybe. Best bet it to try it for your app and see. Mark > -Original Message- > From: Rhuberg,Anthony > Sent: Wednesday, October 09, 2019 2:17 PM > To: users@tomcat.apache.org > Subject: Performance test with Tomcat 9 shows increased cpu/disk usage > because of repeated opening/closing of jars in WEB-INF/lib > > Background: > In the last few months we migrated our web application from Tomcat 7.0.55 to > Tomcat 9.0.19 (26). That transition was relatively straightforward until we > reviewed the results of our performance tests. Those tests showed an increase > in CPU usage and disk I/O on our Windows 2012 server. When we switch back to > Tomcat 7, the metrics for CPU/disk usage were significantly lower (similar to > previous tests). > > As we investigated the cause of the increase in cpu/disk usage we found the > tomcat process was repeatedly opening and closing the jars of our web > application. The process profile showed all the jars in WEB-INF/lib being > closed and reopened within seconds while the application was processing > requests (and to some extent when quite). Now we are trying to determine why. > > We have a general understanding that the WebappClassLoaderBase.java > implementation changed in Tomcat 8.5. The Tomcat 7 implementation provided > for the configuration of jarOpenInterval (which we used the default = 90s). > The latest implementation appears to track the jar modification times with a > hash map called jarModificationTimes, but we are unable to find a similar > time-to-live configuration or what triggers the lifecycle listener to close > the jars (what calls WebappClassLoaderBase.modified() or destroy()) - > assuming we understand this lifecycle behavior. > > The server is configured mostly with default settings. The Context reloadable > property is NOT set. We have tried increasing the cache time of the Resources > cacheTtl property as the default of 5s seemed close to the time the jars were > reloaded. We have not been successful. > > Question: > What is the cause of the repeated reloading of our web application jars? What > can trigger the class loader to close all the file handles related to the > jars in the web application WEB-INF/lib directory - and then open them again > within a few seconds? > > > > CONFIDENTIALITY NOTICE This message and any included attachments are from > Cerner Corporation and are intended only for the addressee. The information > contained in this message is confidential and may constitute inside or > non-public information under international, federal, or state securities > laws. Unauthorized forwarding, printing, copying, distribution, or use of > such information is strictly prohibited and may be unlawful. If you are not > the addressee, please promptly delete this message and notify the sender of > the delivery error by e-mail or you may call Cerner's corporate offices in &g
RE: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
On the other thread: Is this genuine class loading (in which case the response below is correct) or is the application reading the .class files as if they were resources (in which case a different answer applies)? Mostly, we are talking about the default class loader; however, the application does read resources from the jar files - mostly static files that are stored within the package structure are read (for example: log4j, property files, other static content). Would this affect the reload of the jar files? -Original Message- From: Rhuberg,Anthony Sent: Wednesday, October 09, 2019 2:53 PM To: Tomcat Users List Subject: RE: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib Just noticed another thread on this topic: RE: Tomcat discards and reloads the jar files from the webapps folder. Setting the Engine attribute backgroundProcessorDelay to 90s does reduce the open/close cycle of the jars. The other thread also mentioned a potential configuration to keep the jars open longer by default. This could be helpful - not sure we need to close them at all. Alternatively, would there be any gain to 'explode' the jars into class files on the filesystem? -Original Message- From: Rhuberg,Anthony Sent: Wednesday, October 09, 2019 2:17 PM To: users@tomcat.apache.org Subject: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib Background: In the last few months we migrated our web application from Tomcat 7.0.55 to Tomcat 9.0.19 (26). That transition was relatively straightforward until we reviewed the results of our performance tests. Those tests showed an increase in CPU usage and disk I/O on our Windows 2012 server. When we switch back to Tomcat 7, the metrics for CPU/disk usage were significantly lower (similar to previous tests). As we investigated the cause of the increase in cpu/disk usage we found the tomcat process was repeatedly opening and closing the jars of our web application. The process profile showed all the jars in WEB-INF/lib being closed and reopened within seconds while the application was processing requests (and to some extent when quite). Now we are trying to determine why. We have a general understanding that the WebappClassLoaderBase.java implementation changed in Tomcat 8.5. The Tomcat 7 implementation provided for the configuration of jarOpenInterval (which we used the default = 90s). The latest implementation appears to track the jar modification times with a hash map called jarModificationTimes, but we are unable to find a similar time-to-live configuration or what triggers the lifecycle listener to close the jars (what calls WebappClassLoaderBase.modified() or destroy()) - assuming we understand this lifecycle behavior. The server is configured mostly with default settings. The Context reloadable property is NOT set. We have tried increasing the cache time of the Resources cacheTtl property as the default of 5s seemed close to the time the jars were reloaded. We have not been successful. Question: What is the cause of the repeated reloading of our web application jars? What can trigger the class loader to close all the file handles related to the jars in the web application WEB-INF/lib directory - and then open them again within a few seconds? CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
Just noticed another thread on this topic: RE: Tomcat discards and reloads the jar files from the webapps folder. Setting the Engine attribute backgroundProcessorDelay to 90s does reduce the open/close cycle of the jars. The other thread also mentioned a potential configuration to keep the jars open longer by default. This could be helpful - not sure we need to close them at all. Alternatively, would there be any gain to 'explode' the jars into class files on the filesystem? -Original Message- From: Rhuberg,Anthony Sent: Wednesday, October 09, 2019 2:17 PM To: users@tomcat.apache.org Subject: Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib Background: In the last few months we migrated our web application from Tomcat 7.0.55 to Tomcat 9.0.19 (26). That transition was relatively straightforward until we reviewed the results of our performance tests. Those tests showed an increase in CPU usage and disk I/O on our Windows 2012 server. When we switch back to Tomcat 7, the metrics for CPU/disk usage were significantly lower (similar to previous tests). As we investigated the cause of the increase in cpu/disk usage we found the tomcat process was repeatedly opening and closing the jars of our web application. The process profile showed all the jars in WEB-INF/lib being closed and reopened within seconds while the application was processing requests (and to some extent when quite). Now we are trying to determine why. We have a general understanding that the WebappClassLoaderBase.java implementation changed in Tomcat 8.5. The Tomcat 7 implementation provided for the configuration of jarOpenInterval (which we used the default = 90s). The latest implementation appears to track the jar modification times with a hash map called jarModificationTimes, but we are unable to find a similar time-to-live configuration or what triggers the lifecycle listener to close the jars (what calls WebappClassLoaderBase.modified() or destroy()) - assuming we understand this lifecycle behavior. The server is configured mostly with default settings. The Context reloadable property is NOT set. We have tried increasing the cache time of the Resources cacheTtl property as the default of 5s seemed close to the time the jars were reloaded. We have not been successful. Question: What is the cause of the repeated reloading of our web application jars? What can trigger the class loader to close all the file handles related to the jars in the web application WEB-INF/lib directory - and then open them again within a few seconds? CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Performance test with Tomcat 9 shows increased cpu/disk usage because of repeated opening/closing of jars in WEB-INF/lib
Background: In the last few months we migrated our web application from Tomcat 7.0.55 to Tomcat 9.0.19 (26). That transition was relatively straightforward until we reviewed the results of our performance tests. Those tests showed an increase in CPU usage and disk I/O on our Windows 2012 server. When we switch back to Tomcat 7, the metrics for CPU/disk usage were significantly lower (similar to previous tests). As we investigated the cause of the increase in cpu/disk usage we found the tomcat process was repeatedly opening and closing the jars of our web application. The process profile showed all the jars in WEB-INF/lib being closed and reopened within seconds while the application was processing requests (and to some extent when quite). Now we are trying to determine why. We have a general understanding that the WebappClassLoaderBase.java implementation changed in Tomcat 8.5. The Tomcat 7 implementation provided for the configuration of jarOpenInterval (which we used the default = 90s). The latest implementation appears to track the jar modification times with a hash map called jarModificationTimes, but we are unable to find a similar time-to-live configuration or what triggers the lifecycle listener to close the jars (what calls WebappClassLoaderBase.modified() or destroy()) - assuming we understand this lifecycle behavior. The server is configured mostly with default settings. The Context reloadable property is NOT set. We have tried increasing the cache time of the Resources cacheTtl property as the default of 5s seemed close to the time the jars were reloaded. We have not been successful. Question: What is the cause of the repeated reloading of our web application jars? What can trigger the class loader to close all the file handles related to the jars in the web application WEB-INF/lib directory - and then open them again within a few seconds? CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Test for tomcat native
On 04/07/2019 12:46, Markus Fömpe wrote: > Hello, > > I'm not sure if I'm on the right user mailing list or if my question belongs > more on the dev mailing list. If I am wrong here, please let me know. > > Yesterday I got a notification that there is a new version of Tomcat Native > available. I work with macOS and install programs with homebrew. For the last > updates of Tomcat 9 I updated the Homebrew Formula and wanted to do the same > for Tomcat Native [1]. > > During the update of the formula I stumbled over the requirement that now a > test for the formula is necessary [2]. > Unfortunately, I have no idea how such a test could look like for the Tomcat > Native library. Is there someone on the mailing list who could give me a hint? You need a JAVA_HOME to compile the library so compile the Java source as well (a useful test in itself) and write a short Java class that: - calls Library.initialize(null) - checks the version number is correct Mark > > Best regards, > Markus > > [1] > https://github.com/mystygage/homebrew-core/commit/e28e93e3e90c3fa2273990a91c8643ca9c97d296 > [2] https://docs.brew.sh/Formula-Cookbook#add-a-test-to-the-formula > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Test for tomcat native
Hello, I'm not sure if I'm on the right user mailing list or if my question belongs more on the dev mailing list. If I am wrong here, please let me know. Yesterday I got a notification that there is a new version of Tomcat Native available. I work with macOS and install programs with homebrew. For the last updates of Tomcat 9 I updated the Homebrew Formula and wanted to do the same for Tomcat Native [1]. During the update of the formula I stumbled over the requirement that now a test for the formula is necessary [2]. Unfortunately, I have no idea how such a test could look like for the Tomcat Native library. Is there someone on the mailing list who could give me a hint? Best regards, Markus [1] https://github.com/mystygage/homebrew-core/commit/e28e93e3e90c3fa2273990a91c8643ca9c97d296 [2] https://docs.brew.sh/Formula-Cookbook#add-a-test-to-the-formula
Re: Tomcat 8.5 SPNEGO Active Directory stuck with a "Failed authenticate() test"
Hi, I turned on the logging as you recommended, this is what it get in the catalina.out Krb5Context setting mySeqNumber to: 441582303 [Krb5LoginModule]: Entering logout [Krb5LoginModule]: logged out Subject 13-Feb-2019 14:07:56.755 FINE [http-nio-8080-exec-3] org.apache.catalina.authenticator.AuthenticatorBase.invoke Failed authenticate() test 13-Feb-2019 14:07:56.816 FINE [http-nio-8080-exec-5] org.apache.catalina.authenticator.AuthenticatorBase.invoke Security checking request GET /favicon.ico 13-Feb-2019 14:07:56.816 FINE [http-nio-8080-exec-5] org.apache.catalina.realm.RealmBase.findSecurityConstraints Checking constraint 'SecurityConstraint[Tomcat SPNEGO Login Example]' against GET /favicon.ico --> true 13-Feb-2019 14:07:56.817 FINE [http-nio-8080-exec-5] org.apache.catalina.realm.RealmBase.findSecurityConstraints Checking constraint 'SecurityConstraint[Tomcat SPNEGO Login Example]' against GET /favicon.ico --> true 13-Feb-2019 14:07:56.817 FINE [http-nio-8080-exec-5] org.apache.catalina.authenticator.AuthenticatorBase.invoke Calling hasUserDataPermission() 13-Feb-2019 14:07:56.817 FINE [http-nio-8080-exec-5] org.apache.catalina.realm.RealmBase.hasUserDataPermission User data constraint has no restrictions 13-Feb-2019 14:07:56.817 FINE [http-nio-8080-exec-5] org.apache.catalina.authenticator.AuthenticatorBase.invoke Calling authenticate() 13-Feb-2019 14:07:56.818 FINE [http-nio-8080-exec-5] org.apache.catalina.authenticator.SpnegoAuthenticator.doAuthenticate No authorization header sent by client 13-Feb-2019 14:07:56.820 FINE [http-nio-8080-exec-5] org.apache.catalina.authenticator.AuthenticatorBase.invoke Failed authenticate() test The part with the "No authorization header sent by client" seems to be new, however I don't know what to conclude from that information. Kind regards, Thomas Am Mo., 11. Feb. 2019 um 10:24 Uhr schrieb Mark Thomas : > > On 08/02/2019 21:43, Michael Osipov wrote: > > Am 2019-02-08 um 12:54 schrieb Tommy Schneider: > >> Hello, > >> > >> I'm trying to set up Tomcat 8.5 with SPNEGO in the following environment: > >> > >> Tomcat: 8.5.37 built: Dec 12 2018 12:07:02 UTC > >> Platform/OS: AIX 7.2 ppc64 > >> Java: Eclipse OpenJ9 9-internal+0-adhoc.jenkins > >> > >>> From what I can see in the catalina log I think it's almost working > >> (AD user is returned back correctly), but in the web application I > >> always get stuck with a HTTP 401. No matter whether I'm using a JNDI > >> realm or a simple JAAS realm. I also tried different approaches in the > >> application's web.xml like using "*" as generic role name or > >> specifiying a list of role names like they should come back from the > >> AD). I'm starting to think the cause may still be somewhere in the > >> SPNEGO/Kerberos stuff and not in my realm/application config. > >> > >> Currently I'm trying to use a simple JAAS realm (as I found a tutorial > >> saying this is the simplest way to go when you just need the user name > >> and no roles) > >> snippet from server.xml > >> >> autoDeploy="true"> > >> > >> >> className="org.apache.catalina.authenticator.SpnegoAuthenticator" > >> storeDelegatedCredential="true"> > >> >> className="org.apache.catalina.realm.JAASRealm" > >> allRolesMode="strictAuthOnly" /> > >> > >> > >> snippet from catalina.out: > >> Found KeyTab /opt/apache-tomcat-8.5/conf/tomcat.keytab for > >> HTTP/mymachine.mycompany@mycompany.com > >> Found ticket for > >> HTTP/mymachine.mycompany@mycompany.com to go to > >> krbtgt/mycompany@mycompany.com expiring on Fri Feb 08 21:26:27 CET > >> 2019 > >> Entered Krb5Context.acceptSecContext with state=STATE_NEW > >> Looking for keys for: > >> HTTP/mymachine.mycompany@mycompany.com > >> Added key: 17version: 15 > >> Added key: 18version: 15 > >> Added key: 23version: 15 > >> Found unsupported keytype (3) for > >> HTTP/mymachine.mycompany@mycompany.com > >> Found unsupported keytype (1) for > >> HTTP/mymachine.mycompany@mycompany.com > >> >>> EType: > >> sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType > >> Using builtin default etypes for permitted_enctypes > >> d
Re: Tomcat 8.5 SPNEGO Active Directory stuck with a "Failed authenticate() test"
On 08/02/2019 21:43, Michael Osipov wrote: > Am 2019-02-08 um 12:54 schrieb Tommy Schneider: >> Hello, >> >> I'm trying to set up Tomcat 8.5 with SPNEGO in the following environment: >> >> Tomcat: 8.5.37 built: Dec 12 2018 12:07:02 UTC >> Platform/OS: AIX 7.2 ppc64 >> Java: Eclipse OpenJ9 9-internal+0-adhoc.jenkins >> >>> From what I can see in the catalina log I think it's almost working >> (AD user is returned back correctly), but in the web application I >> always get stuck with a HTTP 401. No matter whether I'm using a JNDI >> realm or a simple JAAS realm. I also tried different approaches in the >> application's web.xml like using "*" as generic role name or >> specifiying a list of role names like they should come back from the >> AD). I'm starting to think the cause may still be somewhere in the >> SPNEGO/Kerberos stuff and not in my realm/application config. >> >> Currently I'm trying to use a simple JAAS realm (as I found a tutorial >> saying this is the simplest way to go when you just need the user name >> and no roles) >> snippet from server.xml >> > autoDeploy="true"> >> >> > className="org.apache.catalina.authenticator.SpnegoAuthenticator" >> storeDelegatedCredential="true"> >> > className="org.apache.catalina.realm.JAASRealm" >> allRolesMode="strictAuthOnly" /> >> >> >> snippet from catalina.out: >> Found KeyTab /opt/apache-tomcat-8.5/conf/tomcat.keytab for >> HTTP/mymachine.mycompany@mycompany.com >> Found ticket for >> HTTP/mymachine.mycompany@mycompany.com to go to >> krbtgt/mycompany@mycompany.com expiring on Fri Feb 08 21:26:27 CET >> 2019 >> Entered Krb5Context.acceptSecContext with state=STATE_NEW >> Looking for keys for: >> HTTP/mymachine.mycompany@mycompany.com >> Added key: 17version: 15 >> Added key: 18version: 15 >> Added key: 23version: 15 >> Found unsupported keytype (3) for >> HTTP/mymachine.mycompany@mycompany.com >> Found unsupported keytype (1) for >> HTTP/mymachine.mycompany@mycompany.com >> >>> EType: >> sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType >> Using builtin default etypes for permitted_enctypes >> default etypes for permitted_enctypes: 18 17 16 23. >> >>> EType: >> sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType >> MemoryCache: add >> 1549621587/000784/231A915D0FE70A039CF82095FC685C843F4D981D20A70F972015D8EB16D07CA5/myusern...@mycompany.com >> >> to myusern...@mycompany.com| >> HTTP/mymachine.mycompany@mycompany.com >> >>> KrbApReq: authenticate succeed. >> >>> EType: >> sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType >> >>>Delegated Creds have pname=myusern...@mycompany.com >> sname=krbtgt/mycompany@mycompany.com authtime=null >> starttime=20190208095329Z >> endtime=20190208195235ZrenewTill=20190215095235Z >> Krb5Context setting peerSeqNumber to: 883655442 >> >>> EType: >> sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType >> Krb5Context setting mySeqNumber to: 318684000 >> [Krb5LoginModule]: Entering logout >> [Krb5LoginModule]: logged out Subject >> 08-Feb-2019 11:26:27.415 FINE [https-jsse-nio-443-exec-5] >> org.apache.catalina.authenticator.AuthenticatorBase.invoke Failed >> authenticate() test >> >> I'm happy with the part where "myusern...@mycompany.com" is returned >> back from the AD, so I think most of the stuff is configured correctly >> so far. However I have no idea what the last 3 lines indicate. Is it >> ok that the "logout" occurs here? What causes the authenticator to >> fail? Is this still related to the SPNEGO stuff or is it caused by the >> realm or an incorrect web.xml in the application (I tried different >> variants here and it all seems to end up with a >> "org.apache.catalina.authenticator.AuthenticatorBase.invoke Failed >> authenticate() test". >> >> Let me know if you need more configuration details. >> Any help would be greatly appreciated > > We need more debug outp
Re: Tomcat 8.5 SPNEGO Active Directory stuck with a "Failed authenticate() test"
Am 2019-02-08 um 12:54 schrieb Tommy Schneider: Hello, I'm trying to set up Tomcat 8.5 with SPNEGO in the following environment: Tomcat: 8.5.37 built: Dec 12 2018 12:07:02 UTC Platform/OS: AIX 7.2 ppc64 Java: Eclipse OpenJ9 9-internal+0-adhoc.jenkins From what I can see in the catalina log I think it's almost working (AD user is returned back correctly), but in the web application I always get stuck with a HTTP 401. No matter whether I'm using a JNDI realm or a simple JAAS realm. I also tried different approaches in the application's web.xml like using "*" as generic role name or specifiying a list of role names like they should come back from the AD). I'm starting to think the cause may still be somewhere in the SPNEGO/Kerberos stuff and not in my realm/application config. Currently I'm trying to use a simple JAAS realm (as I found a tutorial saying this is the simplest way to go when you just need the user name and no roles) snippet from server.xml snippet from catalina.out: Found KeyTab /opt/apache-tomcat-8.5/conf/tomcat.keytab for HTTP/mymachine.mycompany@mycompany.com Found ticket for HTTP/mymachine.mycompany@mycompany.com to go to krbtgt/mycompany@mycompany.com expiring on Fri Feb 08 21:26:27 CET 2019 Entered Krb5Context.acceptSecContext with state=STATE_NEW Looking for keys for: HTTP/mymachine.mycompany@mycompany.com Added key: 17version: 15 Added key: 18version: 15 Added key: 23version: 15 Found unsupported keytype (3) for HTTP/mymachine.mycompany@mycompany.com Found unsupported keytype (1) for HTTP/mymachine.mycompany@mycompany.com >>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType Using builtin default etypes for permitted_enctypes default etypes for permitted_enctypes: 18 17 16 23. >>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType MemoryCache: add 1549621587/000784/231A915D0FE70A039CF82095FC685C843F4D981D20A70F972015D8EB16D07CA5/myusern...@mycompany.com to myusern...@mycompany.com| HTTP/mymachine.mycompany@mycompany.com >>> KrbApReq: authenticate succeed. >>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType >>>Delegated Creds have pname=myusern...@mycompany.com sname=krbtgt/mycompany@mycompany.com authtime=null starttime=20190208095329Z endtime=20190208195235ZrenewTill=20190215095235Z Krb5Context setting peerSeqNumber to: 883655442 >>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType Krb5Context setting mySeqNumber to: 318684000 [Krb5LoginModule]: Entering logout [Krb5LoginModule]: logged out Subject 08-Feb-2019 11:26:27.415 FINE [https-jsse-nio-443-exec-5] org.apache.catalina.authenticator.AuthenticatorBase.invoke Failed authenticate() test I'm happy with the part where "myusern...@mycompany.com" is returned back from the AD, so I think most of the stuff is configured correctly so far. However I have no idea what the last 3 lines indicate. Is it ok that the "logout" occurs here? What causes the authenticator to fail? Is this still related to the SPNEGO stuff or is it caused by the realm or an incorrect web.xml in the application (I tried different variants here and it all seems to end up with a "org.apache.catalina.authenticator.AuthenticatorBase.invoke Failed authenticate() test". Let me know if you need more configuration details. Any help would be greatly appreciated We need more debug output. This doesn't really help. Please enable it, it will help. The Kerberos debug output you see is is just Sun-internal which has nothing to do with the Tomcat code. The logout() is performed on the LoginContext required to obtain server credentials. The are released (hence logout performed) as soon as the security context has been established and the GSS src name has been obtained. Michael - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Tomcat 8.5 SPNEGO Active Directory stuck with a "Failed authenticate() test"
Hello, I'm trying to set up Tomcat 8.5 with SPNEGO in the following environment: Tomcat: 8.5.37 built: Dec 12 2018 12:07:02 UTC Platform/OS: AIX 7.2 ppc64 Java: Eclipse OpenJ9 9-internal+0-adhoc.jenkins >From what I can see in the catalina log I think it's almost working (AD user is returned back correctly), but in the web application I always get stuck with a HTTP 401. No matter whether I'm using a JNDI realm or a simple JAAS realm. I also tried different approaches in the application's web.xml like using "*" as generic role name or specifiying a list of role names like they should come back from the AD). I'm starting to think the cause may still be somewhere in the SPNEGO/Kerberos stuff and not in my realm/application config. Currently I'm trying to use a simple JAAS realm (as I found a tutorial saying this is the simplest way to go when you just need the user name and no roles) snippet from server.xml snippet from catalina.out: Found KeyTab /opt/apache-tomcat-8.5/conf/tomcat.keytab for HTTP/mymachine.mycompany@mycompany.com Found ticket for HTTP/mymachine.mycompany@mycompany.com to go to krbtgt/mycompany@mycompany.com expiring on Fri Feb 08 21:26:27 CET 2019 Entered Krb5Context.acceptSecContext with state=STATE_NEW Looking for keys for: HTTP/mymachine.mycompany@mycompany.com Added key: 17version: 15 Added key: 18version: 15 Added key: 23version: 15 Found unsupported keytype (3) for HTTP/mymachine.mycompany@mycompany.com Found unsupported keytype (1) for HTTP/mymachine.mycompany@mycompany.com >>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType Using builtin default etypes for permitted_enctypes default etypes for permitted_enctypes: 18 17 16 23. >>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType MemoryCache: add 1549621587/000784/231A915D0FE70A039CF82095FC685C843F4D981D20A70F972015D8EB16D07CA5/myusern...@mycompany.com to myusern...@mycompany.com| HTTP/mymachine.mycompany@mycompany.com >>> KrbApReq: authenticate succeed. >>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType >>>Delegated Creds have pname=myusern...@mycompany.com sname=krbtgt/mycompany@mycompany.com authtime=null starttime=20190208095329Z endtime=20190208195235ZrenewTill=20190215095235Z Krb5Context setting peerSeqNumber to: 883655442 >>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType Krb5Context setting mySeqNumber to: 318684000 [Krb5LoginModule]: Entering logout [Krb5LoginModule]: logged out Subject 08-Feb-2019 11:26:27.415 FINE [https-jsse-nio-443-exec-5] org.apache.catalina.authenticator.AuthenticatorBase.invoke Failed authenticate() test I'm happy with the part where "myusern...@mycompany.com" is returned back from the AD, so I think most of the stuff is configured correctly so far. However I have no idea what the last 3 lines indicate. Is it ok that the "logout" occurs here? What causes the authenticator to fail? Is this still related to the SPNEGO stuff or is it caused by the realm or an incorrect web.xml in the application (I tried different variants here and it all seems to end up with a "org.apache.catalina.authenticator.AuthenticatorBase.invoke Failed authenticate() test". Let me know if you need more configuration details. Any help would be greatly appreciated thx and kind regards, Thomas - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: test email
2018-06-27 23:34 GMT+03:00 kevin ferguson : > Hi Guys > > Please delete if received I send about 6 emails all bounced not sure why. > The only think I can think is, the mailing list does not like photo > atttachements. There rules are specified here: http://tomcat.apache.org/lists.html#tomcat-users See #7. in that list (no HTML, no attachments). Usually attachments are silently stripped. HTML formatting may cause the message to be treated as spam and be bounced. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
test email
Hi Guys Please delete if received I send about 6 emails all bounced not sure why. The only think I can think is, the mailing list does not like photo atttachements. Kevin
Re: configuring ciphers for SSL Labs server test
Hi Baron, Am 12.05.2018 05:36, schrieb Baron Fujimoto: Hmm, I'm now getting an A grade using: If I'm sufficiently motivated next week, I'll see if I can sort out exactly what the deal was. But for now, it's Friday and pau hana time... (yes, tomcat 8.5.x and Java 1.8_x) On Fri, May 11, 2018 at 07:39:25AM +0100, Mark Thomas wrote: On 11/05/18 03:35, Baron Fujimoto wrote: Yes, the host is behind an F5 load balacer, but AFAIK it should be passing all the TLS/SSL directly to the real host to handle. You don't say which Tomcat version is being used. I assume one of the 8.5.x versions since the 8.5.x docs are referenced. 8.5.x should get an A from SSLLabs with the default configuration: https://wiki.apache.org/tomcat/Security/Ciphers I recently updated that page but 8.5.x was getting a A two years ago as well. Are you sure Java 8 is being used? Mark On Thu, May 10, 2018 at 11:23:44PM +, Scott Hoenigman wrote: Are you using a load balancer? Sent from my T-Mobile 4G LTE Device Original message From: David Wall <d.w...@computer.org> Date: 5/10/18 6:15 PM (GMT-06:00) To: users@tomcat.apache.org Subject: Re: configuring ciphers for SSL Labs server test We're doing good with this: On 5/10/18 2:45 PM, Baron Fujimoto wrote: I'm trying to improve our grade on SSL Labs SSL server test[1] for our Tomcat configuraton. Currently, their report caps our grade at B because, "This server does not support Authenticated encryption (AEAD) cipher suites". They report that we support the following cipher suites: # TLS 1.2 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA # TLS 1.1 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA I'm not sure why SSL Labs is seeing such a limited set of ciphers. We are using Java 1.8.0_162, and I believe we have the Java Cryptography Extension (JCE) properly installed. I have the following connector defined (this version explicitly lists ciphers I think should satisfy the AEAD cipher requirement[2]): protocol="org.apache.coyote.http11.Http11NioProtocol" address="0.0.0.0" port="8443" maxThreads="500" maxPostSize="10" scheme="https" secure="true" defaultSSLHostConfigName="foo.example.edu" SSLEnabled="true" > ciphers="HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK :!TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA :!TLS_DHE_RSA_WITH_AES_128_CBC_SHA :!TLS_DHE_RSA_WITH_AES_256_CBC_SHA :!TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 :!TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 :!TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 :!TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 :!TLS_RSA_WITH_AES_128_CBC_SHA :!TLS_RSA_WITH_AES_256_CBC_SHA :!TLS_RSA_WITH_AES_128_CBC_SHA256 :!TLS_RSA_WITH_AES_256_CBC_SHA256 :!TLS_RSA_WITH_AES_128_GCM_SHA256 :!TLS_RSA_WITH_AES_256_GCM_SHA384 :!TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 :!TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 :TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 :TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384 :TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA :TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA :TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 :TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 :TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 :TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 :TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA :TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA :TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 :TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 :TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 :TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 :TLS_DHE_RSA_WITH_AES_128_CBC_SHA :TLS_DHE_RSA_WITH_AES_256_CBC_SHA :TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 :TLS_DHE_RSA_WITH_AES_256_CBC_
Re: configuring ciphers for SSL Labs server test
Hmm, I'm now getting an A grade using: If I'm sufficiently motivated next week, I'll see if I can sort out exactly what the deal was. But for now, it's Friday and pau hana time... (yes, tomcat 8.5.x and Java 1.8_x) On Fri, May 11, 2018 at 07:39:25AM +0100, Mark Thomas wrote: >On 11/05/18 03:35, Baron Fujimoto wrote: >> Yes, the host is behind an F5 load balacer, but AFAIK it should be passing >> all the TLS/SSL directly to the real host to handle. > >You don't say which Tomcat version is being used. I assume one of the >8.5.x versions since the 8.5.x docs are referenced. > >8.5.x should get an A from SSLLabs with the default configuration: >https://wiki.apache.org/tomcat/Security/Ciphers > >I recently updated that page but 8.5.x was getting a A two years ago as >well. > >Are you sure Java 8 is being used? > >Mark > > >> >> On Thu, May 10, 2018 at 11:23:44PM +, Scott Hoenigman wrote: >>> Are you using a load balancer? >>> >>> >>> >>> Sent from my T-Mobile 4G LTE Device >>> >>> >>> Original message >>> From: David Wall <d.w...@computer.org> >>> Date: 5/10/18 6:15 PM (GMT-06:00) >>> To: users@tomcat.apache.org >>> Subject: Re: configuring ciphers for SSL Labs server test >>> >>> We're doing good with this: >>> >>> >> protocols="TLSv1.1, TLSv1.2" honorCipherOrder="true" >>> ciphers="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, >>> TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, >>> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, >>> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, >>> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, >>> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, >>> TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, >>> TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, >>> TLS_DHE_RSA_WITH_AES_128_CBC_SHA, >>> TLS_DHE_RSA_WITH_AES_256_CBC_SHA, >>> TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, >>> TLS_DHE_RSA_WITH_AES_256_CBC_SHA256" >>> > >>> >>> >>> On 5/10/18 2:45 PM, Baron Fujimoto wrote: >>>> I'm trying to improve our grade on SSL Labs SSL server test[1] for our >>>> Tomcat configuraton. Currently, their report caps our grade at B because, >>>> "This server does not support Authenticated encryption (AEAD) cipher >>>> suites". They report that we support the following cipher suites: >>>> >>>> # TLS 1.2 >>>> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 >>>> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA >>>> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 >>>> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA >>>> >>>> # TLS 1.1 >>>> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA >>>> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA >>>> >>>> I'm not sure why SSL Labs is seeing such a limited set of ciphers. We are >>>> using Java 1.8.0_162, and I believe we have the Java Cryptography >>>> Extension (JCE) properly installed. I have the following connector >>>> defined (this version explicitly lists ciphers I think should satisfy the >>>> AEAD cipher requirement[2]): >>>> >>>> >>> address="0.0.0.0" >>>> port="8443" >>>> maxThreads="500" >>>> maxPostSize="10" >>>> scheme="https" secure="true" >>>> defaultSSLHostConfigName="foo.example.edu" >>>> SSLEnabled="true" > >>>> >>> protocols="TLSv1.1+TLSv1.2+TLS1.3" >>>> certificateVerification="none" >>>> honorCipherOrder="true" >>>> >>>> ciphers="HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK >>>> :!TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA >>>> :!TLS_DHE_RSA_WITH_AES_128_CBC_SHA >>>> :!TLS_DHE_RSA_WITH_AES_256_CBC_SHA >>>> :!TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 >>>> :!TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 >>>> :!TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 >>>> :!TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 >
RE: configuring ciphers for SSL Labs server test
Also, Baron, about the URL you're testing on your site via by SSLLabs: is that really one being served by Tomcat's web server? That's whose connector you're showing here. If instead you are fronting/proxying Tomcat with Apache or IIS, then my understanding is that the SSL support is handled by that web server, not Tomcat (and the connector handling that would be one with a protocol="AJP/1.3" or the like), and you'd then be wanting to really resolve the poor grades via configuration of those instead. I am open to being corrected by you or others here, of course. /charlie >> On 5/10/18 2:45 PM, Baron Fujimoto wrote: >>> I'm trying to improve our grade on SSL Labs SSL server test[1] for >>> our Tomcat configuraton. Currently, their report caps our grade at B >>> because, "This server does not support Authenticated encryption >>> (AEAD) cipher suites". They report that we support the following cipher >>> suites: >>> >>> >>> >> address="0.0.0.0" >>> port="8443" >>> maxThreads="500" >>> maxPostSize="10" >>> scheme="https" secure="true" >>> defaultSSLHostConfigName="foo.example.edu" >>> SSLEnabled="true" > >>> - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: configuring ciphers for SSL Labs server test
On 11/05/18 03:35, Baron Fujimoto wrote: > Yes, the host is behind an F5 load balacer, but AFAIK it should be passing > all the TLS/SSL directly to the real host to handle. You don't say which Tomcat version is being used. I assume one of the 8.5.x versions since the 8.5.x docs are referenced. 8.5.x should get an A from SSLLabs with the default configuration: https://wiki.apache.org/tomcat/Security/Ciphers I recently updated that page but 8.5.x was getting a A two years ago as well. Are you sure Java 8 is being used? Mark > > On Thu, May 10, 2018 at 11:23:44PM +, Scott Hoenigman wrote: >> Are you using a load balancer? >> >> >> >> Sent from my T-Mobile 4G LTE Device >> >> >> Original message >> From: David Wall <d.w...@computer.org> >> Date: 5/10/18 6:15 PM (GMT-06:00) >> To: users@tomcat.apache.org >> Subject: Re: configuring ciphers for SSL Labs server test >> >> We're doing good with this: >> >> > protocols="TLSv1.1, TLSv1.2" honorCipherOrder="true" >> ciphers="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, >> TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, >> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, >> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, >> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, >> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, >> TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, >> TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, >> TLS_DHE_RSA_WITH_AES_128_CBC_SHA, >> TLS_DHE_RSA_WITH_AES_256_CBC_SHA, >> TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, >> TLS_DHE_RSA_WITH_AES_256_CBC_SHA256" >> > >> >> >> On 5/10/18 2:45 PM, Baron Fujimoto wrote: >>> I'm trying to improve our grade on SSL Labs SSL server test[1] for our >>> Tomcat configuraton. Currently, their report caps our grade at B because, >>> "This server does not support Authenticated encryption (AEAD) cipher >>> suites". They report that we support the following cipher suites: >>> >>> # TLS 1.2 >>> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 >>> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA >>> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 >>> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA >>> >>> # TLS 1.1 >>> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA >>> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA >>> >>> I'm not sure why SSL Labs is seeing such a limited set of ciphers. We are >>> using Java 1.8.0_162, and I believe we have the Java Cryptography >>> Extension (JCE) properly installed. I have the following connector >>> defined (this version explicitly lists ciphers I think should satisfy the >>> AEAD cipher requirement[2]): >>> >>> >> address="0.0.0.0" >>> port="8443" >>> maxThreads="500" >>> maxPostSize="10" >>> scheme="https" secure="true" >>> defaultSSLHostConfigName="foo.example.edu" >>> SSLEnabled="true" > >>> >> protocols="TLSv1.1+TLSv1.2+TLS1.3" >>> certificateVerification="none" >>> honorCipherOrder="true" >>> >>> ciphers="HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK >>> :!TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA >>> :!TLS_DHE_RSA_WITH_AES_128_CBC_SHA >>> :!TLS_DHE_RSA_WITH_AES_256_CBC_SHA >>> :!TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 >>> :!TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 >>> :!TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 >>> :!TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 >>> :!TLS_RSA_WITH_AES_128_CBC_SHA >>> :!TLS_RSA_WITH_AES_256_CBC_SHA >>> :!TLS_RSA_WITH_AES_128_CBC_SHA256 >>> :!TLS_RSA_WITH_AES_256_CBC_SHA256 >>> :!TLS_RSA_WITH_AES_128_GCM_SHA256 >>> :!TLS_RSA_WITH_AES_256_GCM_SHA384 >>> :!TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 >>> :!TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 >&
Re: configuring ciphers for SSL Labs server test
Yes, the host is behind an F5 load balacer, but AFAIK it should be passing all the TLS/SSL directly to the real host to handle. On Thu, May 10, 2018 at 11:23:44PM +, Scott Hoenigman wrote: > Are you using a load balancer? > > > >Sent from my T-Mobile 4G LTE Device > > > Original message >From: David Wall <d.w...@computer.org> >Date: 5/10/18 6:15 PM (GMT-06:00) >To: users@tomcat.apache.org >Subject: Re: configuring ciphers for SSL Labs server test > >We're doing good with this: > > protocols="TLSv1.1, TLSv1.2" honorCipherOrder="true" >ciphers="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, >TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, >TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, >TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, >TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, >TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, >TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, >TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, >TLS_DHE_RSA_WITH_AES_128_CBC_SHA, >TLS_DHE_RSA_WITH_AES_256_CBC_SHA, >TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, >TLS_DHE_RSA_WITH_AES_256_CBC_SHA256" > > > > >On 5/10/18 2:45 PM, Baron Fujimoto wrote: >> I'm trying to improve our grade on SSL Labs SSL server test[1] for our >> Tomcat configuraton. Currently, their report caps our grade at B because, >> "This server does not support Authenticated encryption (AEAD) cipher >> suites". They report that we support the following cipher suites: >> >> # TLS 1.2 >> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 >> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA >> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 >> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA >> >> # TLS 1.1 >> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA >> TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA >> >> I'm not sure why SSL Labs is seeing such a limited set of ciphers. We are >> using Java 1.8.0_162, and I believe we have the Java Cryptography >> Extension (JCE) properly installed. I have the following connector >> defined (this version explicitly lists ciphers I think should satisfy the >> AEAD cipher requirement[2]): >> >> > address="0.0.0.0" >> port="8443" >> maxThreads="500" >> maxPostSize="10" >> scheme="https" secure="true" >> defaultSSLHostConfigName="foo.example.edu" >> SSLEnabled="true" > >> > protocols="TLSv1.1+TLSv1.2+TLS1.3" >> certificateVerification="none" >> honorCipherOrder="true" >> >> ciphers="HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK >> :!TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA >> :!TLS_DHE_RSA_WITH_AES_128_CBC_SHA >> :!TLS_DHE_RSA_WITH_AES_256_CBC_SHA >> :!TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 >> :!TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 >> :!TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 >> :!TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 >> :!TLS_RSA_WITH_AES_128_CBC_SHA >> :!TLS_RSA_WITH_AES_256_CBC_SHA >> :!TLS_RSA_WITH_AES_128_CBC_SHA256 >> :!TLS_RSA_WITH_AES_256_CBC_SHA256 >> :!TLS_RSA_WITH_AES_128_GCM_SHA256 >> :!TLS_RSA_WITH_AES_256_GCM_SHA384 >> :!TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 >> :!TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 >> :TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 >> :TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384 >> :TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA >> :TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA >> :TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 >> :TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 >> :TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 >> :TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 >> :TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA >>
Re: configuring ciphers for SSL Labs server test
Are you using a load balancer? Sent from my T-Mobile 4G LTE Device Original message From: David Wall <d.w...@computer.org> Date: 5/10/18 6:15 PM (GMT-06:00) To: users@tomcat.apache.org Subject: Re: configuring ciphers for SSL Labs server test We're doing good with this: On 5/10/18 2:45 PM, Baron Fujimoto wrote: > I'm trying to improve our grade on SSL Labs SSL server test[1] for our > Tomcat configuraton. Currently, their report caps our grade at B because, > "This server does not support Authenticated encryption (AEAD) cipher > suites". They report that we support the following cipher suites: > > # TLS 1.2 > TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 > TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA > TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 > TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA > > # TLS 1.1 > TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA > TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA > > I'm not sure why SSL Labs is seeing such a limited set of ciphers. We are > using Java 1.8.0_162, and I believe we have the Java Cryptography > Extension (JCE) properly installed. I have the following connector > defined (this version explicitly lists ciphers I think should satisfy the > AEAD cipher requirement[2]): > > address="0.0.0.0" > port="8443" > maxThreads="500" > maxPostSize="10" > scheme="https" secure="true" > defaultSSLHostConfigName="foo.example.edu" > SSLEnabled="true" > > protocols="TLSv1.1+TLSv1.2+TLS1.3" > certificateVerification="none" > honorCipherOrder="true" > > ciphers="HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK > :!TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA > :!TLS_DHE_RSA_WITH_AES_128_CBC_SHA > :!TLS_DHE_RSA_WITH_AES_256_CBC_SHA > :!TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 > :!TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 > :!TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 > :!TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 > :!TLS_RSA_WITH_AES_128_CBC_SHA > :!TLS_RSA_WITH_AES_256_CBC_SHA > :!TLS_RSA_WITH_AES_128_CBC_SHA256 > :!TLS_RSA_WITH_AES_256_CBC_SHA256 > :!TLS_RSA_WITH_AES_128_GCM_SHA256 > :!TLS_RSA_WITH_AES_256_GCM_SHA384 > :!TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 > :!TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 > :TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 > :TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384 > :TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA > :TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA > :TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 > :TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 > :TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 > :TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 > :TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA > :TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA > :TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 > :TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 > :TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 > :TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 > :TLS_DHE_RSA_WITH_AES_128_CBC_SHA > :TLS_DHE_RSA_WITH_AES_256_CBC_SHA > :TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 > :TLS_DHE_RSA_WITH_AES_256_CBC_SHA256" > > > certificateKeystoreFile="/home/cas/keystore/foo.pkcs12.keystore" > > > > > > I've mapped the cipher suite names using the OpenSSL cipher suite name > list[3]. I originally started with > ciphers="HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK", but had the same > result, so subsequently tried adding the specific ciphers shown above. The > tomcat SSLHostConfig docs state that either the OpenSSL or
Re: configuring ciphers for SSL Labs server test
We're doing good with this: protocols="TLSv1.1, TLSv1.2" honorCipherOrder="true" ciphers="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256" > On 5/10/18 2:45 PM, Baron Fujimoto wrote: I'm trying to improve our grade on SSL Labs SSL server test[1] for our Tomcat configuraton. Currently, their report caps our grade at B because, "This server does not support Authenticated encryption (AEAD) cipher suites". They report that we support the following cipher suites: # TLS 1.2 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA # TLS 1.1 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA I'm not sure why SSL Labs is seeing such a limited set of ciphers. We are using Java 1.8.0_162, and I believe we have the Java Cryptography Extension (JCE) properly installed. I have the following connector defined (this version explicitly lists ciphers I think should satisfy the AEAD cipher requirement[2]): I've mapped the cipher suite names using the OpenSSL cipher suite name list[3]. I originally started with ciphers="HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK", but had the same result, so subsequently tried adding the specific ciphers shown above. The tomcat SSLHostConfig docs state that either the OpenSSL or JSSE cipher names may be used[4]. Any suggestions on what I may be doing wrong or for further troubleshooting? References: [1] <https://www.ssllabs.com/ssltest/analyze.html> [2] <https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices#23-use-secure-cipher-suites> [3] <https://www.openssl.org/docs/manmaster/man1/ciphers.html#CIPHER-SUITE-NAMES> [4] <https://tomcat.apache.org/tomcat-8.5-doc/config/http.html#SSL_Support_-_SSLHostConfig> - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
configuring ciphers for SSL Labs server test
I'm trying to improve our grade on SSL Labs SSL server test[1] for our Tomcat configuraton. Currently, their report caps our grade at B because, "This server does not support Authenticated encryption (AEAD) cipher suites". They report that we support the following cipher suites: # TLS 1.2 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA # TLS 1.1 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA I'm not sure why SSL Labs is seeing such a limited set of ciphers. We are using Java 1.8.0_162, and I believe we have the Java Cryptography Extension (JCE) properly installed. I have the following connector defined (this version explicitly lists ciphers I think should satisfy the AEAD cipher requirement[2]): I've mapped the cipher suite names using the OpenSSL cipher suite name list[3]. I originally started with ciphers="HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK", but had the same result, so subsequently tried adding the specific ciphers shown above. The tomcat SSLHostConfig docs state that either the OpenSSL or JSSE cipher names may be used[4]. Any suggestions on what I may be doing wrong or for further troubleshooting? References: [1] <https://www.ssllabs.com/ssltest/analyze.html> [2] <https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices#23-use-secure-cipher-suites> [3] <https://www.openssl.org/docs/manmaster/man1/ciphers.html#CIPHER-SUITE-NAMES> [4] <https://tomcat.apache.org/tomcat-8.5-doc/config/http.html#SSL_Support_-_SSLHostConfig> -- Baron Fujimoto <ba...@hawaii.edu> :: UH Information Technology Services minutas cantorum, minutas balorum, minutas carboratum desendus pantorum - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Beginner help setting up test vertical cluster
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Keiichi,
On 11/1/17 3:28 AM, Keiichi Fujino wrote:
> Hi Dave.
>
> Your Interceptor settings are as follows.
>
>
>> > className="org.apache.catalina.tribes.group.
>> interceptors.TcpFailureDetector"> > className="org.apache.catalina.tribes.group.
>> interceptors.ThroughputInterceptor" /> > className="org.apache.catalina.tribes.group.
>> interceptors.StaticMembershipInterceptor"> > className="org.apache. catalina.tribes.membership.StaticMember"
>> domain="clustertest"
>> uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}">
>>
>>
> domain="clustertest" host="xxx.xxx.xxx.xxx" port="4001"
>> securePort="-1"
>> uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0}">
>> > className="org.apache.catalina.tribes.group.
>> interceptors.DomainFilterInterceptor"> > className="org.apache.catalina.tribes.group.
>> interceptors.MessageDispatchInterceptor">
>>
>>
>>
> You specified domain="clustertest" in , but
> DomainFilterInterceptor does not have a domain setting. If you want
> to filter by domain, you have to set domain="clustertest" to
> DomainFilterInterceptor. if you do not want to filter by domain,
> you have to remove domain="clustertest" from or remove
> DomainFilterInterceptor.
>
> Also, if you use DomainFilterInterceptor with static membership,
> you must list it above StaticMembershipInterceptor.
>
> e.g.
> TcpPingInterceptor->TcpFailureDetector->DomainFilterInterceptor->Stati
cMembershipInterceptor
>
>
or
> DomainFilterInterceptor->TcpPingInterceptor->TcpFailureDetector->Stati
cMembershipInterceptor
Would
>
it be appropriate for Tomcat to sanity-check some of the
settings above to catch this kind of oversight? Or are there too many
possibilities of valid configuration that it's not possible to
validate in this way?
Thanks,
- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAln6Ht8ACgkQHPApP6U8
pFhr6hAAtLVR0j+C5UorKkIJv1EIBZJcTS+ZDr/69U89ELLf91OKz0K+6mqmgXl/
TGpkKjKPAhDsmR+1uGh7SR89i0jZVMc29plo1qjd0XIYMJtgViGzA841XFN53Ndh
XI2vIVh+XdunxKO4kWzWaAXdUPVmYkKiZ9A8Sc+c5hYkJA9Pa3PzMZoXWBfhcKHM
hbmGyd1wsFCN+1iWrj2Hd9QAIG9v+gNfJvRQ9e773EOorQz3YMrgjIBUSPy5amC9
E/tng/el9x6jv7FaWIHTH2a12OGZAGSldzDV25w3a3tLq5dIzBGzQytYCPUkPpVr
JOaQewjLLEGgfw39v5EwI4Z/WJAzOP+vozmdYRFyxPgvOwHOkWP7CuXEvHszZD9T
Tggir4TLninyU9Hotq6mzsYeWnaF63OHAPECowOhxro7plZjHdBEqEy+kcn2/r+F
CiGhZI4B7B+RqjK62sx2VNIxNPjiPZycSlt7zXkljkBCRQRWSip1JKcXmQjJgeCB
16mB4rYHSDsAezfvD+/3G20V0j1evl5MPSrn/f9bOj4blOIhMi48m1zEGSyihFr3
SiocXLgje+vXWRXX82IoqH6lR2rcGOwvrGfvO5qzQzT7G7OCRShMZNEBqgWclMOh
tqy/ernLtrp7BuGpJpRn0twqK8qlhGlMyr4kYZ7GIcZEdq7hG/0=
=fSTg
-END PGP SIGNATURE-
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Beginner help setting up test vertical cluster
Hi Dave.
Your Interceptor settings are as follows.
>
>
>
>
>uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}">
>className="org.apache.catalina.tribes.membership.StaticMember"
> domain="clustertest" host="xxx.xxx.xxx.xxx" port="4001" securePort="-1"
> uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0}">
>
>
>
>
>
>
You specified domain="clustertest" in , but DomainFilterInterceptor
does not have a domain setting.
If you want to filter by domain, you have to set domain="clustertest" to
DomainFilterInterceptor.
if you do not want to filter by domain, you have to remove
domain="clustertest" from or remove DomainFilterInterceptor.
Also, if you use DomainFilterInterceptor with static membership, you must
list it above StaticMembershipInterceptor.
e.g.
TcpPingInterceptor->TcpFailureDetector->DomainFilterInterceptor->StaticMembershipInterceptor
or
DomainFilterInterceptor->TcpPingInterceptor->TcpFailureDetector->StaticMembershipInterceptor
--
Keiichi.Fujino
Re: Beginner help setting up test vertical cluster
On Mon, 2017-10-30 at 09:15 -0400, Christopher Schultz wrote: > Dave, > > Can you please post your and associated elements from > conf/server.xml -- minus any secrets that may have crept in there? > Also, what does your network look like? Any intermediates such as > load > balancers/firewalls? What about software firewalls? Hi Chris - thanks for responding. This is all running withing a single machine with no firewall, so nothing should be getting in the way network wise. Here's the my server.xml from one of the instances, with the comments removed and IPs and Hostnames removed also. The IPs/Hostnames are the same on the other server.xml, with just the Member and StaticMember UniqueIDs and port number entries appropriately reversed. Dave --
Re: Beginner help setting up test vertical cluster
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Dave,
On 10/30/17 6:19 AM, Dave Ford wrote:
> Hello,
>
> I should apologise in advance as I'm very new to Tomcat and, I'm
> sure, will be making some daft mistakes and silly errors. I hope
> this question and any that follow it aren't too dumb.
>
> I've recently started a new job and have inherited a half-finished
> tomcat cluster on solaris.
>
> While I'm not going to immediately try and finish that cluster, I
> would like to try setting up some test instances to ensure I know
> what to expect - but I'm having trouble doing even this.
>
> I've gotten my head round how to set up individual instances of
> Tomcat 8.5.23 running from different sub folders of a common
> CATALINA_HOME folder, and now I'm trying to get the two instances
> to talk to each other using StaticMember on specific ports rather
> than the automatic multicasting.
>
> But instances start up, and the logs at least suggest they're aware
> of each other. But one server takes forever to start, and I'm not
> convinced they're talking to each other properly. For example, in
> the catalina.out logs I'm seeing messages such as:
>
>
> 30-Oct-2017 09:28:29.632 INFO [MyHost-startStop-1]
> org.apache.catalina.ha.session.DeltaManager.getAllClusterSessions
> Manager [/sample], requesting session state from
> [org.apache.catalina.tribes.membership.StaticMember[tcp://my.ip.addr:4
0
>
>
01,my.ip.addr,4001, alive=0, securePort=-1, UDP Port=-1, id={1 2 3 4 5
> 6 7 8 9 10 11 12 13 14 15 0 }, payload={}, command={}, domain={99
> 108 117 115 116 101 114 116 101 ...(11)}, ]]. This operation will
> timeout if no session state has been received within [60] seconds.
> 30-Oct-2017 09:28:34.111 WARNING
> [Tribes-Task-Receiver[MyHost-Channel]- 2]
> org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.
m
>
>
essageReceived Received message from
> cluster[org.apache.catalina.tribes.membership.MemberImpl[tcp://{my,
> ip, nn, nn}:4001,{my, ip, nn, nn},4001, alive=1509355714104,
> securePort=- 1, UDP Port=-1, id={1 2 3 4 5 6 7 8 9 10 11 12 13 14
> 15 0 }, payload={}, command={}, domain={99 108 117 115 116 101 114
> 116 101 ...(11)}, ]] was refused.
>
> Which does not seem encouraging. Yet on the other instance, I'm
> seeing:
>
> 30-Oct-2017 09:28:34.105 INFO
> [GroupChannel-Heartbeat[MyHost-Channel]- 1]
> org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded
> Replication member
> added:[org.apache.catalina.tribes.membership.StaticMember[tcp://my.ip.
a
>
>
ddr:4000,my.ip.addr,4000, alive=0, securePort=-1, UDP Port=-1, id={0 1
> 2 3 4 5 6 7 8 9 10 11 12 13 14 15 }, payload={}, command={},
> domain={99 108 117 115 116 101 114 116 101 ...(11)}, ]] 30-Oct-2017
> 09:28:34.105 INFO [GroupChannel-Heartbeat[MyHost-Channel]- 1]
> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.perfo
r
>
>
mBasicCheck Suspect member, confirmed
> alive.[org.apache.catalina.tribes.membership.StaticMember[tcp://my.ip.
a
>
>
ddr:4000,my.ip.addr,4000, alive=0, securePort=-1, UDP Port=-1, id={0 1
> 2 3 4 5 6 7 8 9 10 11 12 13 14 15 }, payload={}, command={},
> domain={99 108 117 115 116 101 114 116 101 ...(11)}, ]]
>
> Which seems better... But then I see back on instance one,
>
> 30-Oct-2017 09:32:30.178 INFO [main]
> org.apache.catalina.startup.Catalina.start Server startup in 241109
> ms
>
> Which seems really bad.
>
> I've installed the 'examples' app on both instances, have added
> '' to both's web.xml file.
>
> I can see on one instance a message such as
>
> 30-Oct-2017 09:35:22.600 INFO [http-nio-8081-exec-7]
> org.apache.catalina.core.ApplicationContext.log SessionListener:
> sessionCreated('C2C3720879E72D558EEF4B0655188EBD.beta') 30-Oct-2017
> 09:35:26.912 INFO [http-nio-8081-exec-8]
> org.apache.catalina.core.ApplicationContext.log SessionListener:
> attributeAdded('C2C3720879E72D558EEF4B0655188EBD.beta', 'foo',
> 'bar')
>
> When I play with the sessions example code. But nothing on the
> other instance that would suggest this is being replicated.
>
> Before I make this post too long and post my server.xml code,
> could someone advise me as to what I should be looking for, or what
> these messages suggest? Any pointers at this stage would be
> grateful as I'm unsure even what a working cluster should look
> like.
Can you please post your and associated elements from
conf/server.xml -- minus any secrets that may have crept in there?
Also, what does your network look like? Any intermediates such as load
balancers/firewalls? What about software firewalls?
- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://w
Beginner help setting up test vertical cluster
Hello,
I should apologise in advance as I'm very new to Tomcat and, I'm sure,
will be making some daft mistakes and silly errors. I hope this
question and any that follow it aren't too dumb.
I've recently started a new job and have inherited a half-finished
tomcat cluster on solaris.
While I'm not going to immediately try and finish that cluster, I would
like to try setting up some test instances to ensure I know what to
expect - but I'm having trouble doing even this.
I've gotten my head round how to set up individual instances of Tomcat
8.5.23 running from different sub folders of a common CATALINA_HOME
folder, and now I'm trying to get the two instances to talk to each
other using StaticMember on specific ports rather than the automatic
multicasting.
But instances start up, and the logs at least suggest they're aware of
each other. But one server takes forever to start, and I'm not
convinced they're talking to each other properly. For example, in the
catalina.out logs I'm seeing messages such as:
30-Oct-2017 09:28:29.632 INFO [MyHost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.getAllClusterSessions
Manager [/sample], requesting session state from
[org.apache.catalina.tribes.membership.StaticMember[tcp://my.ip.addr:40
01,my.ip.addr,4001, alive=0, securePort=-1, UDP Port=-1, id={1 2 3 4 5
6 7 8 9 10 11 12 13 14 15 0 }, payload={}, command={}, domain={99 108
117 115 116 101 114 116 101 ...(11)}, ]]. This operation will timeout
if no session state has been received within [60] seconds.
30-Oct-2017 09:28:34.111 WARNING [Tribes-Task-Receiver[MyHost-Channel]-
2]
org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.m
essageReceived Received message from
cluster[org.apache.catalina.tribes.membership.MemberImpl[tcp://{my, ip,
nn, nn}:4001,{my, ip, nn, nn},4001, alive=1509355714104, securePort=-
1,
UDP Port=-1, id={1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 }, payload={},
command={}, domain={99 108 117 115 116 101 114 116 101 ...(11)}, ]] was
refused.
Which does not seem encouraging. Yet on the other instance, I'm
seeing:
30-Oct-2017 09:28:34.105 INFO [GroupChannel-Heartbeat[MyHost-Channel]-
1] org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication
member
added:[org.apache.catalina.tribes.membership.StaticMember[tcp://my.ip.a
ddr:4000,my.ip.addr,4000, alive=0, securePort=-1, UDP Port=-1, id={0 1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 }, payload={}, command={}, domain={99
108 117 115 116 101 114 116 101 ...(11)}, ]]
30-Oct-2017 09:28:34.105 INFO [GroupChannel-Heartbeat[MyHost-Channel]-
1]
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.perfor
mBasicCheck Suspect member, confirmed
alive.[org.apache.catalina.tribes.membership.StaticMember[tcp://my.ip.a
ddr:4000,my.ip.addr,4000, alive=0, securePort=-1, UDP Port=-1, id={0 1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 }, payload={}, command={}, domain={99
108 117 115 116 101 114 116 101 ...(11)}, ]]
Which seems better... But then I see back on instance one,
30-Oct-2017 09:32:30.178 INFO [main]
org.apache.catalina.startup.Catalina.start Server startup in 241109 ms
Which seems really bad.
I've installed the 'examples' app on both instances, have added
'' to both's web.xml file.
I can see on one instance a message such as
30-Oct-2017 09:35:22.600 INFO [http-nio-8081-exec-7]
org.apache.catalina.core.ApplicationContext.log SessionListener:
sessionCreated('C2C3720879E72D558EEF4B0655188EBD.beta')
30-Oct-2017 09:35:26.912 INFO [http-nio-8081-exec-8]
org.apache.catalina.core.ApplicationContext.log SessionListener:
attributeAdded('C2C3720879E72D558EEF4B0655188EBD.beta', 'foo', 'bar')
When I play with the sessions example code. But nothing on the other
instance that would suggest this is being replicated.
Before I make this post too long and post my server.xml code, could
someone advise me as to what I should be looking for, or what these
messages suggest? Any pointers at this stage would be grateful as I'm
unsure even what a working cluster should look like.
Thanks
Dave
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Join WebEx meeting in progress: Test v2
On 03/08/17 12:09, Mark Thomas wrote: > Hello, > My WebEx meeting is in progress. Sorry folks. PLease ignore this. Mark > > > > *Test v2* > Thursday, 3 August 2017 > 12:09 | GMT Summer Time (London, GMT+01:00) | 1 hr > > Meeting number: 314 070 218 > > > Join Meeting > <https://pivotal.webex.com/pivotal/e.php?MTID=mf67360b0c9cb219d96a2000a2fcfd729> > > > > > > > > > > Can't join the meeting? <https://help.webex.com/docs/DOC-5412> > > > > IMPORTANT NOTICE: Please note that this WebEx service allows audio and > other information sent during the session to be recorded, which may be > discoverable in a legal matter. By joining this session, you > automatically consent to such recordings. If you do not consent to being > recorded, discuss your concerns with the host or do not join the session. > > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Join WebEx meeting in progress: Test v2
Hello, My WebEx meeting is in progress. Test v2 Thursday, 3 August 2017 12:09 | GMT Summer Time (London, GMT+01:00) | 1 hr Meeting number: 314 070 218 Join Meeting https://pivotal.webex.com/pivotal/e.php?MTID=mf67360b0c9cb219d96a2000a2fcfd729 Can't join the meeting? https://help.webex.com/docs/DOC-5412 IMPORTANT NOTICE: Please note that this WebEx service allows audio and other information sent during the session to be recorded, which may be discoverable in a legal matter. By joining this session, you automatically consent to such recordings. If you do not consent to being recorded, discuss your concerns with the host or do not join the session. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: tomcat ant test failure due to java exception.
On 02/05/2017 04:24, Arjit Gupta wrote: > Hi Mark, > > Below is the full stack trace of errors. > > java.lang.AssertionError: Option not found > at sun.nio.ch.Net.setSocketOption(Net.java:352) > at > sun.nio.ch.SocketChannelImpl.setOption(SocketChannelImpl.java:179) > at sun.nio.ch.SocketAdaptor.setIntOption(SocketAdaptor.java:305) > at sun.nio.ch.SocketAdaptor.setTrafficClass(SocketAdaptor.java:405) That method is part of the public API for java.net.Socket and has been since Java 1.4. The only documented exceptions are SocketException and IllegalArgumentException. AssertionError should not be thrown there. That is a JRE bug and should be reported to your JRE vendor. > at > org.apache.catalina.tribes.transport.nio.NioSender.configureSocket(NioSender.java:146) Since this is happening during the tests, you can exclude the failing tests use the test.exclude property. See BUILDING.txt for details. Mark > > Arjit Kumar > > On Sun, Apr 30, 2017 at 4:36 PM, Mark Thomas <ma...@apache.org> wrote: > >> On 29/04/17 03:47, Arjit Gupta wrote: >>> Hi, >>> >>> I am building tomcat 7.0.77 with java7 on HP-UX(OS) itanium(IA64) >> processor. >>> After building it successful I am running *ant test*. >>> Multiple tests are failing due to below exception. >>> >>> *Exception in thread "Thread-18" java.lang.AssertionError: Option not >> found* >> >> Full stack trace for one of those errors? >> >> Mark >> >> >>> >>> Fail test cases are:- >>> 175 >>> TEST-org.apache.catalina.tribes.group.TestGroupChannelSenderConnecti >> ons.BIO.txt:Tests >>> run: 3, Failures: 3, Errors: 0, Time elapsed: 9.213 sec >>> 176 >>> TEST-org.apache.catalina.tribes.group.TestGroupChannelSenderConnecti >> ons.NIO.txt:Tests >>> run: 3, Failures: 3, Errors: 0, Time elapsed: 9.572 sec >>> 183 >>> TEST-org.apache.catalina.tribes.group.interceptors. >> TestNonBlockingCoordinator.BIO.txt:Tests >>> run: 2, Failures: 2, Errors: 0, Time elapsed: 43.762 sec >>> 184 >>> TEST-org.apache.catalina.tribes.group.interceptors. >> TestNonBlockingCoordinator.NIO.txt:Tests >>> run: 2, Failures: 2, Errors: 0, Time elapsed: 43.897 sec >>> 185 >>> TEST-org.apache.catalina.tribes.group.interceptors. >> TestOrderInterceptor.BIO.txt:Tests >>> run: 2, Failures: 2, Errors: 0, Time elapsed: 30.797 sec >>> 186 >>> TEST-org.apache.catalina.tribes.group.interceptors. >> TestOrderInterceptor.NIO.txt:Tests >>> run: 2, Failures: 2, Errors: 0, Time elapsed: 31.078 sec >>> 187 >>> TEST-org.apache.catalina.tribes.group.interceptors. >> TestTcpFailureDetector.BIO.txt:Tests >>> run: 3, Failures: 2, Errors: 0, Time elapsed: 27.097 sec >>> 188 >>> TEST-org.apache.catalina.tribes.group.interceptors. >> TestTcpFailureDetector.NIO.txt:Tests >>> run: 3, Failures: 2, Errors: 0, Time elapsed: 33.172 sec >>> 359 TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt:Tests run: >> 5, >>> Failures: 0, Errors: 5, Time elapsed: 7.45 sec >>> 360 TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt:Tests run: >> 5, >>> Failures: 0, Errors: 5, Time elapsed: 8.081 sec >>> 361 TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt:Tests run: 3, >>> Failures: 0, Errors: 3, Time elapsed: 7.54 sec >>> 362 TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt:Tests run: 3, >>> Failures: 0, Errors: 3, Time elapsed: 8.156 sec >>> 363 TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Tests run: 4, >>> Failures: 0, Errors: 3, Time elapsed: 8.117 sec >>> 364 TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Tests run: 4, >>> Failures: 0, Errors: 3, Time elapsed: 8.933 sec >>> >>> Arjit Kumar >>> >> >> >> - >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: tomcat ant test failure due to java exception.
Hi Mark, Below is the full stack trace of errors. java.lang.AssertionError: Option not found at sun.nio.ch.Net.setSocketOption(Net.java:352) at sun.nio.ch.SocketChannelImpl.setOption(SocketChannelImpl.java:179) at sun.nio.ch.SocketAdaptor.setIntOption(SocketAdaptor.java:305) at sun.nio.ch.SocketAdaptor.setTrafficClass(SocketAdaptor.java:405) at org.apache.catalina.tribes.transport.nio.NioSender.configureSocket(NioSender.java:146) at org.apache.catalina.tribes.transport.nio.NioSender.connect(NioSender.java:247) at org.apache.catalina.tribes.transport.nio.ParallelNioSender.connect(ParallelNioSender.java:201) at org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMessage(ParallelNioSender.java:78) at org.apache.catalina.tribes.transport.nio.PooledParallelSender.sendMessage(PooledParallelSender.java:54) at org.apache.catalina.tribes.transport.ReplicationTransmitter.sendMessage(ReplicationTransmitter.java:81) at org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage(ChannelCoordinator.java:79) at org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:79) at org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:79) at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.sendMessage(TcpFailureDetector.java:93) at org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:79) at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.sendElectionMsg(NonBlockingCoordinator.java:255) at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.startElection(NonBlockingCoordinator.java:222) at org.apache.catalina.tribes.group.interceptors.NonBlockingCoordinator.memberDisappeared(NonBlockingCoordinator.java:545) at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:100) at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.memberDisappeared(TcpFailureDetector.java:158) at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:100) at org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.memberDisappeared(DomainFilterInterceptor.java:84) at org.apache.catalina.tribes.group.ChannelInterceptorBase.memberDisappeared(ChannelInterceptorBase.java:100) at org.apache.catalina.tribes.group.ChannelCoordinator.memberDisappeared(ChannelCoordinator.java:267) at org.apache.catalina.tribes.membership.McastService.memberDisappeared(McastService.java:562) at org.apache.catalina.tribes.membership.McastServiceImpl$1.run(McastServiceImpl.java:376) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Arjit Kumar On Sun, Apr 30, 2017 at 4:36 PM, Mark Thomas <ma...@apache.org> wrote: > On 29/04/17 03:47, Arjit Gupta wrote: > > Hi, > > > > I am building tomcat 7.0.77 with java7 on HP-UX(OS) itanium(IA64) > processor. > > After building it successful I am running *ant test*. > > Multiple tests are failing due to below exception. > > > > *Exception in thread "Thread-18" java.lang.AssertionError: Option not > found* > > Full stack trace for one of those errors? > > Mark > > > > > > Fail test cases are:- > > 175 > > TEST-org.apache.catalina.tribes.group.TestGroupChannelSenderConnecti > ons.BIO.txt:Tests > > run: 3, Failures: 3, Errors: 0, Time elapsed: 9.213 sec > > 176 > > TEST-org.apache.catalina.tribes.group.TestGroupChannelSenderConnecti > ons.NIO.txt:Tests > > run: 3, Failures: 3, Errors: 0, Time elapsed: 9.572 sec > > 183 > > TEST-org.apache.catalina.tribes.group.interceptors. > TestNonBlockingCoordinator.BIO.txt:Tests > > run: 2, Failures: 2, Errors: 0, Time elapsed: 43.762 sec > > 184 > > TEST-org.apache.catalina.tribes.group.interceptors. > TestNonBlockingCoordinator.NIO.txt:Tests > > run: 2, Failures: 2, Errors: 0, Time elapsed: 43.897 sec > > 185 > > TEST-org.apache.catalina.tribes.group.interceptors. > TestOrderInterceptor.BIO.txt:Tests > > run: 2, Failures: 2, Errors: 0, Time elapsed: 30.797 sec > > 186 > > TEST-org.apache.catalina.tribes.group.interceptors. > TestOrderInterceptor.NIO.txt:Tests > > run: 2, Failures: 2, Errors: 0, Time elapsed: 31.078 sec > > 187 > > TEST-org.apache.catalina.tribes.group.interceptors. > TestTcpFailureDetector.BIO.txt:Tests &
Re: tomcat ant test failure due to java exception.
On 29/04/17 03:47, Arjit Gupta wrote: > Hi, > > I am building tomcat 7.0.77 with java7 on HP-UX(OS) itanium(IA64) processor. > After building it successful I am running *ant test*. > Multiple tests are failing due to below exception. > > *Exception in thread "Thread-18" java.lang.AssertionError: Option not found* Full stack trace for one of those errors? Mark > > Fail test cases are:- > 175 > > TEST-org.apache.catalina.tribes.group.TestGroupChannelSenderConnections.BIO.txt:Tests > run: 3, Failures: 3, Errors: 0, Time elapsed: 9.213 sec > 176 > > TEST-org.apache.catalina.tribes.group.TestGroupChannelSenderConnections.NIO.txt:Tests > run: 3, Failures: 3, Errors: 0, Time elapsed: 9.572 sec > 183 > > TEST-org.apache.catalina.tribes.group.interceptors.TestNonBlockingCoordinator.BIO.txt:Tests > run: 2, Failures: 2, Errors: 0, Time elapsed: 43.762 sec > 184 > > TEST-org.apache.catalina.tribes.group.interceptors.TestNonBlockingCoordinator.NIO.txt:Tests > run: 2, Failures: 2, Errors: 0, Time elapsed: 43.897 sec > 185 > > TEST-org.apache.catalina.tribes.group.interceptors.TestOrderInterceptor.BIO.txt:Tests > run: 2, Failures: 2, Errors: 0, Time elapsed: 30.797 sec > 186 > > TEST-org.apache.catalina.tribes.group.interceptors.TestOrderInterceptor.NIO.txt:Tests > run: 2, Failures: 2, Errors: 0, Time elapsed: 31.078 sec > 187 > > TEST-org.apache.catalina.tribes.group.interceptors.TestTcpFailureDetector.BIO.txt:Tests > run: 3, Failures: 2, Errors: 0, Time elapsed: 27.097 sec > 188 > > TEST-org.apache.catalina.tribes.group.interceptors.TestTcpFailureDetector.NIO.txt:Tests > run: 3, Failures: 2, Errors: 0, Time elapsed: 33.172 sec > 359 TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt:Tests run: 5, > Failures: 0, Errors: 5, Time elapsed: 7.45 sec > 360 TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt:Tests run: 5, > Failures: 0, Errors: 5, Time elapsed: 8.081 sec > 361 TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt:Tests run: 3, > Failures: 0, Errors: 3, Time elapsed: 7.54 sec > 362 TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt:Tests run: 3, > Failures: 0, Errors: 3, Time elapsed: 8.156 sec > 363 TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Tests run: 4, > Failures: 0, Errors: 3, Time elapsed: 8.117 sec > 364 TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Tests run: 4, > Failures: 0, Errors: 3, Time elapsed: 8.933 sec > > Arjit Kumar > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
tomcat ant test failure due to java exception.
Hi, I am building tomcat 7.0.77 with java7 on HP-UX(OS) itanium(IA64) processor. After building it successful I am running *ant test*. Multiple tests are failing due to below exception. *Exception in thread "Thread-18" java.lang.AssertionError: Option not found* Fail test cases are:- 175 TEST-org.apache.catalina.tribes.group.TestGroupChannelSenderConnections.BIO.txt:Tests run: 3, Failures: 3, Errors: 0, Time elapsed: 9.213 sec 176 TEST-org.apache.catalina.tribes.group.TestGroupChannelSenderConnections.NIO.txt:Tests run: 3, Failures: 3, Errors: 0, Time elapsed: 9.572 sec 183 TEST-org.apache.catalina.tribes.group.interceptors.TestNonBlockingCoordinator.BIO.txt:Tests run: 2, Failures: 2, Errors: 0, Time elapsed: 43.762 sec 184 TEST-org.apache.catalina.tribes.group.interceptors.TestNonBlockingCoordinator.NIO.txt:Tests run: 2, Failures: 2, Errors: 0, Time elapsed: 43.897 sec 185 TEST-org.apache.catalina.tribes.group.interceptors.TestOrderInterceptor.BIO.txt:Tests run: 2, Failures: 2, Errors: 0, Time elapsed: 30.797 sec 186 TEST-org.apache.catalina.tribes.group.interceptors.TestOrderInterceptor.NIO.txt:Tests run: 2, Failures: 2, Errors: 0, Time elapsed: 31.078 sec 187 TEST-org.apache.catalina.tribes.group.interceptors.TestTcpFailureDetector.BIO.txt:Tests run: 3, Failures: 2, Errors: 0, Time elapsed: 27.097 sec 188 TEST-org.apache.catalina.tribes.group.interceptors.TestTcpFailureDetector.NIO.txt:Tests run: 3, Failures: 2, Errors: 0, Time elapsed: 33.172 sec 359 TEST-org.apache.tomcat.util.net.TestClientCert.BIO.txt:Tests run: 5, Failures: 0, Errors: 5, Time elapsed: 7.45 sec 360 TEST-org.apache.tomcat.util.net.TestClientCert.NIO.txt:Tests run: 5, Failures: 0, Errors: 5, Time elapsed: 8.081 sec 361 TEST-org.apache.tomcat.util.net.TestCustomSsl.BIO.txt:Tests run: 3, Failures: 0, Errors: 3, Time elapsed: 7.54 sec 362 TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO.txt:Tests run: 3, Failures: 0, Errors: 3, Time elapsed: 8.156 sec 363 TEST-org.apache.tomcat.util.net.TestSsl.BIO.txt:Tests run: 4, Failures: 0, Errors: 3, Time elapsed: 8.117 sec 364 TEST-org.apache.tomcat.util.net.TestSsl.NIO.txt:Tests run: 4, Failures: 0, Errors: 3, Time elapsed: 8.933 sec Arjit Kumar
Re: Tomcat not scaling under load - Tsung v Apache Bench test
Thanks for feedback. On 29.11.2016 15:43, Szymon Czaja wrote: Hi Andre, I have not run all my tests yet but it looks like you were correct. The Amazon load balancer appears to have been the culprit. I will let you know if it turns out otherwise. Thank you for pushing me to take the load balancer out. Cheers! Szymon On 28 November 2016 at 13:45, Szymon Czaja <szymonpiotrcz...@gmail.com> wrote: Completely agree. I am in the process of setting up a direct connection to Tomcat bypassing load balancer. Will give an update soon. Szymon On 28 November 2016 at 13:40, André Warnier (tomcat) <a...@ice-sa.com> wrote: On 28.11.2016 14:10, Szymon Czaja wrote: I will run a test in the meantime to check but this is very much unlikely, I can see on the Amazon console that the load balancer CPU is hardly doing anything with peaks at 2%. Let's assume load balancer is not an issue. I am not saying that the proxy cannot handle it. But maybe it is limiting the number of connections that it forwards, if they are all coming from the same client IP ? (Avoid DoS attacks, that kind of thing..) Still a guess. But netstat on both sides may be telling you more. If it is tomcat that could not handle the load, then you'd probably still see the same number of connections on the tomcat server side, at the TCP level. That tomcat would be able to "service" these connections is another matter, but the connections themselves should be there. Szymon On 28 November 2016 at 12:29, André Warnier (tomcat) <a...@ice-sa.com> wrote: On 28.11.2016 12:52, Szymon Czaja wrote: Hi, I have updated my question on SO. I have noticed that the number of ESTABLISHED connections goes up on the client after few minutes. I expected the same on the server which does not seem to be the case. Any ideas? Just a guess without looking very deep into your data : you say somewhere that the requests go through a proxy. Maybe the client "established" TCP connections are with that proxy, while the ones you see on the tomcat server are the connections from the proxy ? (In other words, it is the proxy that is the bottleneck ?) Szymon On 28 November 2016 at 10:05, Mark Thomas <ma...@apache.org> wrote: On 28/11/2016 09:53, Szymon Czaja wrote: Hi, I have asked the question on StackOverflow but I am not getting much response: http://stackoverflow.com/questions/40793335/why-tomcat- does-not-pass-a-tsung-performance-test-at-40-requests-per-second Could anyone help me understand why is Tomcat unable to keep up with the processing when running Tsung yet Apache Bench tests do not relveal any scalability issues? I'd recommend taking some thread dumps and looking at netstat output to try and figure out what is going on. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat not scaling under load - Tsung v Apache Bench test
Hi Andre, I have not run all my tests yet but it looks like you were correct. The Amazon load balancer appears to have been the culprit. I will let you know if it turns out otherwise. Thank you for pushing me to take the load balancer out. Cheers! Szymon On 28 November 2016 at 13:45, Szymon Czaja <szymonpiotrcz...@gmail.com> wrote: > Completely agree. I am in the process of setting up a direct connection to > Tomcat bypassing load balancer. Will give an update soon. > > Szymon > > On 28 November 2016 at 13:40, André Warnier (tomcat) <a...@ice-sa.com> > wrote: > >> On 28.11.2016 14:10, Szymon Czaja wrote: >> >>> I will run a test in the meantime to check but this is very much >>> unlikely, >>> I can see on the Amazon console that the load balancer CPU is hardly >>> doing >>> anything with peaks at 2%. Let's assume load balancer is not an issue. >>> >> >> I am not saying that the proxy cannot handle it. But maybe it is limiting >> the number of connections that it forwards, if they are all coming from the >> same client IP ? >> (Avoid DoS attacks, that kind of thing..) >> Still a guess. But netstat on both sides may be telling you more. >> >> If it is tomcat that could not handle the load, then you'd probably still >> see the same number of connections on the tomcat server side, at the TCP >> level. >> That tomcat would be able to "service" these connections is another >> matter, but the connections themselves should be there. >> >> >> >>> Szymon >>> >>> On 28 November 2016 at 12:29, André Warnier (tomcat) <a...@ice-sa.com> >>> wrote: >>> >>> On 28.11.2016 12:52, Szymon Czaja wrote: >>>> >>>> Hi, >>>>> I have updated my question on SO. I have noticed that the number of >>>>> ESTABLISHED connections goes up on the client after few minutes. I >>>>> expected >>>>> the same on the server which does not seem to be the case. Any ideas? >>>>> >>>>> >>>> Just a guess without looking very deep into your data : you say >>>> somewhere >>>> that the requests go through a proxy. Maybe the client "established" TCP >>>> connections are with that proxy, while the ones you see on the tomcat >>>> server are the connections from the proxy ? >>>> (In other words, it is the proxy that is the bottleneck ?) >>>> >>>> >>>> >>>> >>>> Szymon >>>>> >>>>> On 28 November 2016 at 10:05, Mark Thomas <ma...@apache.org> wrote: >>>>> >>>>> On 28/11/2016 09:53, Szymon Czaja wrote: >>>>> >>>>>> >>>>>> Hi, >>>>>>> I have asked the question on StackOverflow but I am not getting much >>>>>>> response: >>>>>>> >>>>>>> http://stackoverflow.com/questions/40793335/why-tomcat- >>>>>>> does-not-pass-a-tsung-performance-test-at-40-requests-per-second >>>>>>> >>>>>>> Could anyone help me understand why is Tomcat unable to keep up with >>>>>>> the >>>>>>> processing when running Tsung yet Apache Bench tests do not relveal >>>>>>> any >>>>>>> scalability issues? >>>>>>> >>>>>>> >>>>>> I'd recommend taking some thread dumps and looking at netstat output >>>>>> to >>>>>> try and figure out what is going on. >>>>>> >>>>>> Mark >>>>>> >>>>>> >>>>>> - >>>>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>>>> For additional commands, e-mail: users-h...@tomcat.apache.org >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> - >>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>> For additional commands, e-mail: users-h...@tomcat.apache.org >>>> >>>> >>>> >>> >> >> - >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> >
Re: Tomcat not scaling under load - Tsung v Apache Bench test
Completely agree. I am in the process of setting up a direct connection to Tomcat bypassing load balancer. Will give an update soon. Szymon On 28 November 2016 at 13:40, André Warnier (tomcat) <a...@ice-sa.com> wrote: > On 28.11.2016 14:10, Szymon Czaja wrote: > >> I will run a test in the meantime to check but this is very much unlikely, >> I can see on the Amazon console that the load balancer CPU is hardly doing >> anything with peaks at 2%. Let's assume load balancer is not an issue. >> > > I am not saying that the proxy cannot handle it. But maybe it is limiting > the number of connections that it forwards, if they are all coming from the > same client IP ? > (Avoid DoS attacks, that kind of thing..) > Still a guess. But netstat on both sides may be telling you more. > > If it is tomcat that could not handle the load, then you'd probably still > see the same number of connections on the tomcat server side, at the TCP > level. > That tomcat would be able to "service" these connections is another > matter, but the connections themselves should be there. > > > >> Szymon >> >> On 28 November 2016 at 12:29, André Warnier (tomcat) <a...@ice-sa.com> >> wrote: >> >> On 28.11.2016 12:52, Szymon Czaja wrote: >>> >>> Hi, >>>> I have updated my question on SO. I have noticed that the number of >>>> ESTABLISHED connections goes up on the client after few minutes. I >>>> expected >>>> the same on the server which does not seem to be the case. Any ideas? >>>> >>>> >>> Just a guess without looking very deep into your data : you say somewhere >>> that the requests go through a proxy. Maybe the client "established" TCP >>> connections are with that proxy, while the ones you see on the tomcat >>> server are the connections from the proxy ? >>> (In other words, it is the proxy that is the bottleneck ?) >>> >>> >>> >>> >>> Szymon >>>> >>>> On 28 November 2016 at 10:05, Mark Thomas <ma...@apache.org> wrote: >>>> >>>> On 28/11/2016 09:53, Szymon Czaja wrote: >>>> >>>>> >>>>> Hi, >>>>>> I have asked the question on StackOverflow but I am not getting much >>>>>> response: >>>>>> >>>>>> http://stackoverflow.com/questions/40793335/why-tomcat- >>>>>> does-not-pass-a-tsung-performance-test-at-40-requests-per-second >>>>>> >>>>>> Could anyone help me understand why is Tomcat unable to keep up with >>>>>> the >>>>>> processing when running Tsung yet Apache Bench tests do not relveal >>>>>> any >>>>>> scalability issues? >>>>>> >>>>>> >>>>> I'd recommend taking some thread dumps and looking at netstat output to >>>>> try and figure out what is going on. >>>>> >>>>> Mark >>>>> >>>>> >>>>> - >>>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>>> For additional commands, e-mail: users-h...@tomcat.apache.org >>>>> >>>>> >>>>> >>>>> >>>> >>> - >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: users-h...@tomcat.apache.org >>> >>> >>> >> > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Tomcat not scaling under load - Tsung v Apache Bench test
On 28.11.2016 14:10, Szymon Czaja wrote: I will run a test in the meantime to check but this is very much unlikely, I can see on the Amazon console that the load balancer CPU is hardly doing anything with peaks at 2%. Let's assume load balancer is not an issue. I am not saying that the proxy cannot handle it. But maybe it is limiting the number of connections that it forwards, if they are all coming from the same client IP ? (Avoid DoS attacks, that kind of thing..) Still a guess. But netstat on both sides may be telling you more. If it is tomcat that could not handle the load, then you'd probably still see the same number of connections on the tomcat server side, at the TCP level. That tomcat would be able to "service" these connections is another matter, but the connections themselves should be there. Szymon On 28 November 2016 at 12:29, André Warnier (tomcat) <a...@ice-sa.com> wrote: On 28.11.2016 12:52, Szymon Czaja wrote: Hi, I have updated my question on SO. I have noticed that the number of ESTABLISHED connections goes up on the client after few minutes. I expected the same on the server which does not seem to be the case. Any ideas? Just a guess without looking very deep into your data : you say somewhere that the requests go through a proxy. Maybe the client "established" TCP connections are with that proxy, while the ones you see on the tomcat server are the connections from the proxy ? (In other words, it is the proxy that is the bottleneck ?) Szymon On 28 November 2016 at 10:05, Mark Thomas <ma...@apache.org> wrote: On 28/11/2016 09:53, Szymon Czaja wrote: Hi, I have asked the question on StackOverflow but I am not getting much response: http://stackoverflow.com/questions/40793335/why-tomcat- does-not-pass-a-tsung-performance-test-at-40-requests-per-second Could anyone help me understand why is Tomcat unable to keep up with the processing when running Tsung yet Apache Bench tests do not relveal any scalability issues? I'd recommend taking some thread dumps and looking at netstat output to try and figure out what is going on. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat not scaling under load - Tsung v Apache Bench test
I will run a test in the meantime to check but this is very much unlikely, I can see on the Amazon console that the load balancer CPU is hardly doing anything with peaks at 2%. Let's assume load balancer is not an issue. Szymon On 28 November 2016 at 12:29, André Warnier (tomcat) <a...@ice-sa.com> wrote: > On 28.11.2016 12:52, Szymon Czaja wrote: > >> Hi, >> I have updated my question on SO. I have noticed that the number of >> ESTABLISHED connections goes up on the client after few minutes. I >> expected >> the same on the server which does not seem to be the case. Any ideas? >> > > Just a guess without looking very deep into your data : you say somewhere > that the requests go through a proxy. Maybe the client "established" TCP > connections are with that proxy, while the ones you see on the tomcat > server are the connections from the proxy ? > (In other words, it is the proxy that is the bottleneck ?) > > > > >> Szymon >> >> On 28 November 2016 at 10:05, Mark Thomas <ma...@apache.org> wrote: >> >> On 28/11/2016 09:53, Szymon Czaja wrote: >>> >>>> Hi, >>>> I have asked the question on StackOverflow but I am not getting much >>>> response: >>>> >>>> http://stackoverflow.com/questions/40793335/why-tomcat- >>>> does-not-pass-a-tsung-performance-test-at-40-requests-per-second >>>> >>>> Could anyone help me understand why is Tomcat unable to keep up with the >>>> processing when running Tsung yet Apache Bench tests do not relveal any >>>> scalability issues? >>>> >>> >>> I'd recommend taking some thread dumps and looking at netstat output to >>> try and figure out what is going on. >>> >>> Mark >>> >>> >>> - >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: users-h...@tomcat.apache.org >>> >>> >>> >> > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Tomcat not scaling under load - Tsung v Apache Bench test
On 28.11.2016 12:52, Szymon Czaja wrote: Hi, I have updated my question on SO. I have noticed that the number of ESTABLISHED connections goes up on the client after few minutes. I expected the same on the server which does not seem to be the case. Any ideas? Just a guess without looking very deep into your data : you say somewhere that the requests go through a proxy. Maybe the client "established" TCP connections are with that proxy, while the ones you see on the tomcat server are the connections from the proxy ? (In other words, it is the proxy that is the bottleneck ?) Szymon On 28 November 2016 at 10:05, Mark Thomas <ma...@apache.org> wrote: On 28/11/2016 09:53, Szymon Czaja wrote: Hi, I have asked the question on StackOverflow but I am not getting much response: http://stackoverflow.com/questions/40793335/why-tomcat- does-not-pass-a-tsung-performance-test-at-40-requests-per-second Could anyone help me understand why is Tomcat unable to keep up with the processing when running Tsung yet Apache Bench tests do not relveal any scalability issues? I'd recommend taking some thread dumps and looking at netstat output to try and figure out what is going on. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat not scaling under load - Tsung v Apache Bench test
Hi, I have updated my question on SO. I have noticed that the number of ESTABLISHED connections goes up on the client after few minutes. I expected the same on the server which does not seem to be the case. Any ideas? Szymon On 28 November 2016 at 10:05, Mark Thomas <ma...@apache.org> wrote: > On 28/11/2016 09:53, Szymon Czaja wrote: > > Hi, > > I have asked the question on StackOverflow but I am not getting much > > response: > > > > http://stackoverflow.com/questions/40793335/why-tomcat- > > does-not-pass-a-tsung-performance-test-at-40-requests-per-second > > > > Could anyone help me understand why is Tomcat unable to keep up with the > > processing when running Tsung yet Apache Bench tests do not relveal any > > scalability issues? > > I'd recommend taking some thread dumps and looking at netstat output to > try and figure out what is going on. > > Mark > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Tomcat not scaling under load - Tsung v Apache Bench test
On 28/11/2016 09:53, Szymon Czaja wrote: > Hi, > I have asked the question on StackOverflow but I am not getting much > response: > > http://stackoverflow.com/questions/40793335/why-tomcat- > does-not-pass-a-tsung-performance-test-at-40-requests-per-second > > Could anyone help me understand why is Tomcat unable to keep up with the > processing when running Tsung yet Apache Bench tests do not relveal any > scalability issues? I'd recommend taking some thread dumps and looking at netstat output to try and figure out what is going on. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Tomcat not scaling under load - Tsung v Apache Bench test
Hi, I have asked the question on StackOverflow but I am not getting much response: http://stackoverflow.com/questions/40793335/why-tomcat- does-not-pass-a-tsung-performance-test-at-40-requests-per-second Could anyone help me understand why is Tomcat unable to keep up with the processing when running Tsung yet Apache Bench tests do not relveal any scalability issues? Thank you, Szymon
Re: Performance degrades on tomcat7 for the same runs of a sample performance 60 user test
Dimple, On 11/9/15 3:46 AM, dimple ranka wrote: > Some help is really appreciated, atleast it will be good to hear if some > one else is also facing slowness with the tomcat7 server. > > I did some more investigation here and as mentioned earlier saw the > slowness with tomcat7 is more reproducible when there are some > cryptographic signature verifications. I was going to ask about the crypto. One of the only threads actually doing anything in the thread dump you showed was showing a TLS handshake. Java's crypto is not very fast compared to native crypto. IF you are handling TLS from Tomcat, you might want to consider a faster solution if you do a lot of transactions. If you have a shortage of entropy on the server, generating new crypto keys can really slow things down. That could certainly explain the "first run is fast, later runs are slower" observation. But this has absolutely nothing to do with Tomcat -- it's all within the JVM so you shouldn't have noticed a change with a Tomcat upgrade. > Turning on the StuckThreadDetectionValve with 1 second threshold as > shown below revealed that there are warning logs with threads stuck > showing up in the next run of the test. This clearly shows that the > slowness is due to the threads getting into the blocked state. More > are in the signature verification but few are in other areas too. > > className="org.apache.catalina.valves.StuckThreadDetectionValve" > threshold="1" /> > > NOTE - the above test was performed on a low end windows machine. Since the > granularity of the valve threshold is in seconds was finding it hard to > reproduce the valve to spit out some logs on the high end server. But the > test shows that in the second run of the performance test which was run on > another machine, only then i see the warning logs on the tomcat server. > > [snip] > > WARNING: Thread "http-nio-8443-exec-55" (id=83) has been active for 1,397 > milliseconds (since 11/7/15 11:06 AM) to serve the same request for > https://10.55.198.52:8443/Signature/authenticate and may be stuck > (configured threshold for this StuckThreadDetectionValve is 1 seconds). > There is/are 1 thread(s) in total that are monitored by this Valve and may > be stuck. > java.lang.Throwable > at sun.security.ec.ECDSASignature.verifySignedDigest(Native Method) > at sun.security.ec.ECDSASignature.engineVerify(ECDSASignature.java:309) > at java.security.Signature$Delegate.engineVerify(Signature.java:1192) > at java.security.Signature.verify(Signature.java:626) > at signature.SignatureEndPoint.verifyECDSASignature(Unknown Source) > at signature.SignatureEndPoint.verifySignature(Unknown Source) > at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) > at > com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) > at > com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) > at > com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) > at > com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) > at > com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) > at > com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) > at > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511) > at > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442) > at > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391) > at > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381) > at > com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) > at > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538) > at > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at org.ap
Re: Performance degrades on tomcat7 for the same runs of a sample performance 60 user test
On Mon, Nov 9, 2015 at 8:34 PM, Christopher Schultz < ch...@christopherschultz.net> wrote: > Dimple, > > On 11/9/15 3:46 AM, dimple ranka wrote: > > Some help is really appreciated, atleast it will be good to hear if some > > one else is also facing slowness with the tomcat7 server. > > > > I did some more investigation here and as mentioned earlier saw the > > slowness with tomcat7 is more reproducible when there are some > > cryptographic signature verifications. > > I was going to ask about the crypto. One of the only threads actually > doing anything in the thread dump you showed was showing a TLS > handshake. Java's crypto is not very fast compared to native crypto. IF > you are handling TLS from Tomcat, you might want to consider a faster > solution if you do a lot of transactions. > > If you have a shortage of entropy on the server, generating new crypto > keys can really slow things down. That could certainly explain the > "first run is fast, later runs are slower" observation. But this has > absolutely nothing to do with Tomcat -- it's all within the JVM so you > shouldn't have noticed a change with a Tomcat upgrade. > > ==> We have configured the secure random source as below to avoid the entropy issue - securerandom.source=file:/dev/./urandom > Turning on the StuckThreadDetectionValve with 1 second threshold as > > shown below revealed that there are warning logs with threads stuck > > showing up in the next run of the test. This clearly shows that the > > slowness is due to the threads getting into the blocked state. More > > are in the signature verification but few are in other areas too. > > > > > className="org.apache.catalina.valves.StuckThreadDetectionValve" > > threshold="1" /> > > > > NOTE - the above test was performed on a low end windows machine. Since > the > > granularity of the valve threshold is in seconds was finding it hard to > > reproduce the valve to spit out some logs on the high end server. But the > > test shows that in the second run of the performance test which was run > on > > another machine, only then i see the warning logs on the tomcat server. > > > > [snip] > > > > > WARNING: Thread "http-nio-8443-exec-55" (id=83) has been active for 1,397 > > milliseconds (since 11/7/15 11:06 AM) to serve the same request for > > https://10.55.198.52:8443/Signature/authenticate and may be stuck > > (configured threshold for this StuckThreadDetectionValve is 1 seconds). > > There is/are 1 thread(s) in total that are monitored by this Valve and > may > > be stuck. > > java.lang.Throwable > > at sun.security.ec.ECDSASignature.verifySignedDigest(Native Method) > > at sun.security.ec.ECDSASignature.engineVerify(ECDSASignature.java:309) > > at java.security.Signature$Delegate.engineVerify(Signature.java:1192) > > at java.security.Signature.verify(Signature.java:626) > > at signature.SignatureEndPoint.verifyECDSASignature(Unknown Source) > > at signature.SignatureEndPoint.verifySignature(Unknown Source) > > at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:606) > > at > > > com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) > > at > > > com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) > > at > > > com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) > > at > > > com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) > > at > > > com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) > > at > > > com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) > > at > > > com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) > > at > > > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511) > > at > > > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442) > > at > > > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391) > > at > > > com.sun.jersey.server.impl.application.WebApplicationI
Re: Performance degrades on tomcat7 for the same runs of a sample performance 60 user test
Hi all, Some help is really appreciated, atleast it will be good to hear if some one else is also facing slowness with the tomcat7 server. I did some more investigation here and as mentioned earlier saw the slowness with tomcat7 is more reproducible when there are some cryptographic signature verifications. Turning on the StuckThreadDetectionValve with 1 second threshold as shown below revealed that there are warning logs with threads stuck showing up in the next run of the test. This clearly shows that the slowness is due to the threads getting into the blocked state. More are in the signature verification but few are in other areas too. NOTE - the above test was performed on a low end windows machine. Since the granularity of the valve threshold is in seconds was finding it hard to reproduce the valve to spit out some logs on the high end server. But the test shows that in the second run of the performance test which was run on another machine, only then i see the warning logs on the tomcat server. CATALINA.out contents showing the stuck threads ===> Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig undeploy INFO: Undeploying context [/host-manager] Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig undeploy INFO: Undeploying context [/docs] Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig undeploy INFO: Undeploying context [] Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig undeploy INFO: Undeploying context [/Signature] Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig undeploy INFO: Undeploying context [/examples] Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig undeploy INFO: Undeploying context [/manager] Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig deployWAR INFO: Deploying web application archive D:\apache-tomcat-7.0.64\webapps\Signature.war Nov 07, 2015 11:01:16 AM org.apache.catalina.startup.TldConfig execute INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time. Nov 07, 2015 11:01:17 AM com.sun.jersey.api.core.servlet.WebAppResourceConfig init INFO: Scanning for root resource and provider classes in the Web app resource paths: /WEB-INF/lib /WEB-INF/classes Nov 07, 2015 11:01:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses INFO: Root resource classes found: class signature.NullEndPoint class signature.SignatureEndPoint Nov 07, 2015 11:01:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses INFO: Provider classes found: class org.codehaus.jackson.jaxrs.JsonMappingExceptionMapper class org.codehaus.jackson.jaxrs.JacksonJaxbJsonProvider class org.codehaus.jackson.jaxrs.JacksonJsonProvider class org.codehaus.jackson.jaxrs.JsonParseExceptionMapper Nov 07, 2015 11:01:18 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.17 01/17/2013 03:31 PM' Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig deployWAR INFO: Deployment of web application archive D:\apache-tomcat-7.0.64\webapps\Signature.war has finished in 5,871 ms Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory D:\apache-tomcat-7.0.64\webapps\docs Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.TldConfig execute INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time. Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deployment of web application directory D:\apache-tomcat-7.0.64\webapps\docs has finished in 140 ms Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory D:\apache-tomcat-7.0.64\webapps\examples Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.TldConfig execute INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time. Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deployment of web application directory D:\apache-tomcat-7.0.64\webapps\examples has finished in 445 ms Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory D:\apache-tomcat-7.0.64\webapps\host-manager Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.TldConfig execute INFO: At least one JAR was scan
Re: Performance degrades on tomcat7 for the same runs of a sample performance 60 user test
Looks like attachments are not allowed so only sharing the contents of threadDump3.out 2015-11-04 22:55:58 Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.79-b02 mixed mode): "http-nio-8443-exec-147" daemon prio=10 tid=0x7f5c1c005000 nid=0x6241 runnable [0x7f5bd30ef000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:192) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) - locked <0xfc2d9498> (a java.lang.Object) at org.apache.tomcat.util.net.SecureNioChannel.read(SecureNioChannel.java:443) at org.apache.coyote.http11.InternalNioInputBuffer.readSocket(InternalNioInputBuffer.java:436) at org.apache.coyote.http11.InternalNioInputBuffer.fill(InternalNioInputBuffer.java:795) at org.apache.coyote.http11.InternalNioInputBuffer.parseRequestLine(InternalNioInputBuffer.java:227) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:993) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719) - locked <0x875077f0> (a org.apache.tomcat.util.net.SecureNioChannel) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Locked ownable synchronizers: - <0x88a613f0> (a java.util.concurrent.ThreadPoolExecutor$Worker) "http-nio-8443-exec-146" daemon prio=10 tid=0x7f5c18020800 nid=0x6240 waiting on condition [0x7f5c6e271000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x86b7a088> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86) at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Locked ownable synchronizers: - None "http-nio-8443-exec-145" daemon prio=10 tid=0x7f5c18037000 nid=0x623f waiting on condition [0x7f5c6e372000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x86b7a088> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86) at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Locked ownable synchronizers: - None "http-nio-8443-exec-144" daemon prio=10 tid=0x7f5c1802f800 nid=0x623e waiting on condition [0x7f5c6c89c000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x86b7a088> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86) at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at
Re: Performance degrades on tomcat7 for the same runs of a sample performance 60 user test
Hi Chris, I captured 3 threadumps with the below requests/second when the server is already slow. We were using BIO before and had observed slowness even with that. We switched to using NIO since it seemed a recommendation for production environment. -- *WITH NIO* *--* *RUN1 (threadDump1.out)* Requests/second = 18849.1606557377 *RUN2 **(threadDump2.out)* Requests/second = 18943.43606557377 *RUN3 **(threadDump3.out)* Requests/second = 18894.21241830065 -- The issue we are trying to address why does performance degrade by 50% with subsequent runs. Thanks, Dimple On Thu, Nov 5, 2015 at 1:31 AM, Christopher Schultz < ch...@christopherschultz.net> wrote: > Dimple, > > On 11/3/15 11:24 PM, dimple ranka wrote: > > Also forgot to mention that setting maxKeepAliveRequests to -1 did not > > help. As expected from the connector documentation the default value for > > this attribute is 100 and we have a 60 user test set up. > > > > On Wed, Nov 4, 2015 at 8:18 AM, dimple ranka <dimplekra...@gmail.com> > wrote: > > > >> Hi Mark, > >> > >> Another observation to be noted here is that system CPU usage shoots up > >> for subsequent runs, especially for later runs. > >> > >> We have been looking into this issue for couple of weeks now and it is > >> clear in the same environment with the same setup it doesn't reproduce > on > >> tomcat6. The moment we deploy the web application in a tomcat7 container > >> the slowness with subsequent runs shows up. > > Can you take some thread dumps to find out what Tomcat is doing? High > CPU usage and lower request throughput sounds like a Poller problem > since you are using NIO. > > -chris > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance degrades on tomcat7 for the same runs of a sample performance 60 user test
Dimple, On 11/3/15 11:24 PM, dimple ranka wrote: > Also forgot to mention that setting maxKeepAliveRequests to -1 did not > help. As expected from the connector documentation the default value for > this attribute is 100 and we have a 60 user test set up. > > On Wed, Nov 4, 2015 at 8:18 AM, dimple ranka <dimplekra...@gmail.com> wrote: > >> Hi Mark, >> >> Another observation to be noted here is that system CPU usage shoots up >> for subsequent runs, especially for later runs. >> >> We have been looking into this issue for couple of weeks now and it is >> clear in the same environment with the same setup it doesn't reproduce on >> tomcat6. The moment we deploy the web application in a tomcat7 container >> the slowness with subsequent runs shows up. Can you take some thread dumps to find out what Tomcat is doing? High CPU usage and lower request throughput sounds like a Poller problem since you are using NIO. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance degrades on tomcat7 for the same runs of a sample performance 60 user test
On 03/11/2015 06:17, dimple ranka wrote: > ## > On tomcat7 number of requests fullfilled per second > ## > * RUN1 ** ==> ** 38128.97704918033 runs/sec* > *RUN2 ==> ** 19040.35947712418 runs/sec* > *RUN3 ==> ** 19043.7908496732** runs/sec* > * RUN4 ==> ** 19001.71568627451 runs/sec* > ## Every performance test I have ever done with Tomcat, the first run has had poorer performance while the system warms up (threads started, caches filled, session ID generators initialised etc.) and subsequent tests have performed better. The results above suggest something about the test is broken. First of all, thank you for the detailed information. It helps a lot. Your server side test code won't run on Tomcat without additional libraries and configuration that you haven't specified. That makes it harder for people to reproduce your results. I'd recommend using a simple servlet for testing in the first instance and only when any issues have been resolved, move to your test. 20k requests/s is rather low. I'd expect those sorts of numbers from a single threaded test on reasonable hardware. With 8*2 threads to play with on the server I'd expect that number to be a lot higher. My experience of JMeter is that the results are unreliable as you approach the point where the overhead per request for JMeter approaches the overhead per request for Tomcat. You would probably get better results if you switched to a lower overhead test client. I tend to use ab for these sorts of tests. It is not clear from your original e-mail if the test client is running on the same machine as the server or not. If not, the network can have a significant impact. With this test a gigabit network should be fine but we have seen network saturation in performance tests for larger static files so do keep an eye on the network utilisation just to be safe. The higher the throughput, the greater the impact of HTTP keep-alive. I have frequently observed strange test results caused by the system running out of free ports due to the churn rate of connections. netstat is your friend. I'd recommend exploring the impact of different settings for maxKeepAliveRequests. You almost certainly want something significantly higher than the default of 100. A quick test would be to set it to -1 and see what impact that has. HTH, Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance degrades on tomcat7 for the same runs of a sample performance 60 user test
Hi Mark, The test client is running on another machine. The server side test code is written using jersey 1.17. Will try out with playing with maxKeepAliveRequests. My only concern is that if jmeter is doing spoofy things then why it doesnt show up on tomcat6. Thanks, Dimple On Tue, Nov 3, 2015 at 2:27 PM, Mark Thomas <ma...@apache.org> wrote: > On 03/11/2015 06:17, dimple ranka wrote: > > > > > ## > > On tomcat7 number of requests fullfilled per second > > ## > > * RUN1 ** ==> ** 38128.97704918033 runs/sec* > > *RUN2 ==> ** 19040.35947712418 runs/sec* > > *RUN3 ==> ** 19043.7908496732** runs/sec* > > * RUN4 ==> ** 19001.71568627451 runs/sec* > > ## > > Every performance test I have ever done with Tomcat, the first run has > had poorer performance while the system warms up (threads started, > caches filled, session ID generators initialised etc.) and subsequent > tests have performed better. > > The results above suggest something about the test is broken. > > First of all, thank you for the detailed information. It helps a lot. > > Your server side test code won't run on Tomcat without additional > libraries and configuration that you haven't specified. That makes it > harder for people to reproduce your results. I'd recommend using a > simple servlet for testing in the first instance and only when any > issues have been resolved, move to your test. > > 20k requests/s is rather low. I'd expect those sorts of numbers from a > single threaded test on reasonable hardware. With 8*2 threads to play > with on the server I'd expect that number to be a lot higher. > > My experience of JMeter is that the results are unreliable as you > approach the point where the overhead per request for JMeter approaches > the overhead per request for Tomcat. You would probably get better > results if you switched to a lower overhead test client. I tend to use > ab for these sorts of tests. > > It is not clear from your original e-mail if the test client is running > on the same machine as the server or not. > If not, the network can have a significant impact. With this test a > gigabit network should be fine but we have seen network saturation in > performance tests for larger static files so do keep an eye on the > network utilisation just to be safe. > > The higher the throughput, the greater the impact of HTTP keep-alive. I > have frequently observed strange test results caused by the system > running out of free ports due to the churn rate of connections. netstat > is your friend. I'd recommend exploring the impact of different settings > for maxKeepAliveRequests. You almost certainly want something > significantly higher than the default of 100. A quick test would be to > set it to -1 and see what impact that has. > > HTH, > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Performance degrades on tomcat7 for the same runs of a sample performance 60 user test
Hi Mark, Another observation to be noted here is that system CPU usage shoots up for subsequent runs, especially for later runs. We have been looking into this issue for couple of weeks now and it is clear in the same environment with the same setup it doesn't reproduce on tomcat6. The moment we deploy the web application in a tomcat7 container the slowness with subsequent runs shows up. Thanks, Dimple On Tue, Nov 3, 2015 at 2:36 PM, dimple ranka <dimplekra...@gmail.com> wrote: > Hi Mark, > > The test client is running on another machine. > > The server side test code is written using jersey 1.17. > > Will try out with playing with maxKeepAliveRequests. > > My only concern is that if jmeter is doing spoofy things then why it > doesnt show up on tomcat6. > > Thanks, > Dimple > > On Tue, Nov 3, 2015 at 2:27 PM, Mark Thomas <ma...@apache.org> wrote: > >> On 03/11/2015 06:17, dimple ranka wrote: >> >> >> >> > ## >> > On tomcat7 number of requests fullfilled per second >> > ## >> > * RUN1 ** ==> ** 38128.97704918033 runs/sec* >> > *RUN2 ==> ** 19040.35947712418 runs/sec* >> > *RUN3 ==> ** 19043.7908496732** runs/sec* >> > * RUN4 ==> ** 19001.71568627451 runs/sec* >> > ## >> >> Every performance test I have ever done with Tomcat, the first run has >> had poorer performance while the system warms up (threads started, >> caches filled, session ID generators initialised etc.) and subsequent >> tests have performed better. >> >> The results above suggest something about the test is broken. >> >> First of all, thank you for the detailed information. It helps a lot. >> >> Your server side test code won't run on Tomcat without additional >> libraries and configuration that you haven't specified. That makes it >> harder for people to reproduce your results. I'd recommend using a >> simple servlet for testing in the first instance and only when any >> issues have been resolved, move to your test. >> >> 20k requests/s is rather low. I'd expect those sorts of numbers from a >> single threaded test on reasonable hardware. With 8*2 threads to play >> with on the server I'd expect that number to be a lot higher. >> >> My experience of JMeter is that the results are unreliable as you >> approach the point where the overhead per request for JMeter approaches >> the overhead per request for Tomcat. You would probably get better >> results if you switched to a lower overhead test client. I tend to use >> ab for these sorts of tests. >> >> It is not clear from your original e-mail if the test client is running >> on the same machine as the server or not. >> If not, the network can have a significant impact. With this test a >> gigabit network should be fine but we have seen network saturation in >> performance tests for larger static files so do keep an eye on the >> network utilisation just to be safe. >> >> The higher the throughput, the greater the impact of HTTP keep-alive. I >> have frequently observed strange test results caused by the system >> running out of free ports due to the churn rate of connections. netstat >> is your friend. I'd recommend exploring the impact of different settings >> for maxKeepAliveRequests. You almost certainly want something >> significantly higher than the default of 100. A quick test would be to >> set it to -1 and see what impact that has. >> >> HTH, >> >> Mark >> >> - >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> >
Re: Performance degrades on tomcat7 for the same runs of a sample performance 60 user test
Also forgot to mention that setting maxKeepAliveRequests to -1 did not help. As expected from the connector documentation the default value for this attribute is 100 and we have a 60 user test set up. On Wed, Nov 4, 2015 at 8:18 AM, dimple ranka <dimplekra...@gmail.com> wrote: > Hi Mark, > > Another observation to be noted here is that system CPU usage shoots up > for subsequent runs, especially for later runs. > > We have been looking into this issue for couple of weeks now and it is > clear in the same environment with the same setup it doesn't reproduce on > tomcat6. The moment we deploy the web application in a tomcat7 container > the slowness with subsequent runs shows up. > > Thanks, > Dimple > > On Tue, Nov 3, 2015 at 2:36 PM, dimple ranka <dimplekra...@gmail.com> > wrote: > >> Hi Mark, >> >> The test client is running on another machine. >> >> The server side test code is written using jersey 1.17. >> >> Will try out with playing with maxKeepAliveRequests. >> >> My only concern is that if jmeter is doing spoofy things then why it >> doesnt show up on tomcat6. >> >> Thanks, >> Dimple >> >> On Tue, Nov 3, 2015 at 2:27 PM, Mark Thomas <ma...@apache.org> wrote: >> >>> On 03/11/2015 06:17, dimple ranka wrote: >>> >>> >>> >>> > ## >>> > On tomcat7 number of requests fullfilled per second >>> > ## >>> > * RUN1 ** ==> ** 38128.97704918033 runs/sec* >>> > *RUN2 ==> ** 19040.35947712418 runs/sec* >>> > *RUN3 ==> ** 19043.7908496732** runs/sec* >>> > * RUN4 ==> ** 19001.71568627451 runs/sec* >>> > ## >>> >>> Every performance test I have ever done with Tomcat, the first run has >>> had poorer performance while the system warms up (threads started, >>> caches filled, session ID generators initialised etc.) and subsequent >>> tests have performed better. >>> >>> The results above suggest something about the test is broken. >>> >>> First of all, thank you for the detailed information. It helps a lot. >>> >>> Your server side test code won't run on Tomcat without additional >>> libraries and configuration that you haven't specified. That makes it >>> harder for people to reproduce your results. I'd recommend using a >>> simple servlet for testing in the first instance and only when any >>> issues have been resolved, move to your test. >>> >>> 20k requests/s is rather low. I'd expect those sorts of numbers from a >>> single threaded test on reasonable hardware. With 8*2 threads to play >>> with on the server I'd expect that number to be a lot higher. >>> >>> My experience of JMeter is that the results are unreliable as you >>> approach the point where the overhead per request for JMeter approaches >>> the overhead per request for Tomcat. You would probably get better >>> results if you switched to a lower overhead test client. I tend to use >>> ab for these sorts of tests. >>> >>> It is not clear from your original e-mail if the test client is running >>> on the same machine as the server or not. >>> If not, the network can have a significant impact. With this test a >>> gigabit network should be fine but we have seen network saturation in >>> performance tests for larger static files so do keep an eye on the >>> network utilisation just to be safe. >>> >>> The higher the throughput, the greater the impact of HTTP keep-alive. I >>> have frequently observed strange test results caused by the system >>> running out of free ports due to the churn rate of connections. netstat >>> is your friend. I'd recommend exploring the impact of different settings >>> for maxKeepAliveRequests. You almost certainly want something >>> significantly higher than the default of 100. A quick test would be to >>> set it to -1 and see what impact that has. >>> >>> HTH, >>> >>> Mark >>> >>> - >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: users-h...@tomcat.apache.org >>> >>> >> >
Performance degrades on tomcat7 for the same runs of a sample performance 60 user test
Hi all,
I am doing some performance data collection on tomcat7 on a 8 processor
high end machine with CentOS release 6.7 (Final) operating system and i am
observing slowness with subsequent runs of the same jmeter test. Here are
the version and specfication details followed by the endpoint source and
the problem statement finally.
##
Tomcat7 Version ==>
##
Server version: Apache Tomcat/7.0.64
Server built: Aug 19 2015 17:18:06 UTC
Server number:* 7.0.64.0*
OS Name:Linux
OS Version: 2.6.32-573.3.1.el6.x86_64
Architecture: amd64
JVM Version:1.7.0_79-b15
JVM Vendor: Oracle Corporation
##
##
Connector configurations
##
##
##
Machine Specifications
##
Architecture: x86_64
CPU op-mode(s):32-bit, 64-bit
Byte Order:Little Endian
CPU(s):8
On-line CPU(s) list: 0-7
Thread(s) per core:2
Core(s) per socket:4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family:6
Model: 58
Stepping: 9
CPU MHz: 1600.000
BogoMIPS: 7000.55
Virtualization:VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 8192K
NUMA node0 CPU(s): 0-7
##
##
Source code of the endpoint being performance tested
##
import javax.ws.rs.Consumes;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
@Path("/null")
public class NullEndPoint {
public static final String UTF_8 = "UTF-8";
public static final String JSON_CONTENT_TYPE = MediaType.APPLICATION_JSON +
"; charset=" + UTF_8;
@POST
@Consumes(JSON_CONTENT_TYPE)
@Produces(JSON_CONTENT_TYPE)
public Response verify(String params) {
return Response.ok("{ \"status\":\"success\"}").build();
}
}
##
##
Jmeter Test Plan - 60 users test for 1200 seconds duration
##
false
false
continue
${__P(users,60)}
0
10
0
30
10
1
${__P(duration,1200)}
5
false
-1
Content-Type
application/json
userNameUUID
user_${__UUID
=
host
${__P(myhost,XXX.XX.XX.XX)}
=
port
8443
=
protocol
https
=
pathPrefix
/null
=
true
false
{
}
=
${host}
${port}
${protocol}
${pathPrefix}
POST
true
false
true
false
false
false
saveConfig
true
true
true
true
true
true
true
true
true
false
true
true
false
false
false
false
false
false
false
false
0
true
##
##
On tomcat7 number of requests fullfilled per second
##
* RUN1 ** ==> ** 38128.97704918033 runs/sec*
*RUN2 ==> ** 19040.35947712418 runs/sec*
*RUN3 ==> ** 19043.7908496732** runs/sec*
* RUN4 ==> ** 19001.71568627451 runs/sec*
##
##
*PRO
Re: SPNEGO test configuration with Manager webapp
On 14/05/2015 22:29, Mark Thomas wrote: On 14/05/2015 21:11, Mark Thomas wrote: On 29/03/2015 23:13, André Warnier wrote: David Marsh wrote: I've tested all the following public JDKs jdk-7u45-windows-i586.exe jdk-7u65-windows-i586.exe jdk-7u75-windows-i586.exe jdk-8-windows-i586.exe jdk-8u5-windows-i586.exe jdk-8u11-windows-i586.exe jdk-8u20-windows-i586.exe jdk-8u25-windows-i586.exe jdk-8u31-windows-i586.exe jdk-8u40-windows-i586.exe -- Only this one fails SPNEGO / Bad GSS Token Seems a recent fix must broken it. That is really great info. Thanks. As promised I have found some time to look into this. It appears that this fix in 8u40 onwards broke SPNEGO. https://bugs.openjdk.java.net/browse/JDK-8048194 The fix that was applied wasn't the one suggested in the bug report. I've spent some time looking at the code but I haven't found a way around this yet. Good news (sort of). I have an *extremely* dirty hack that fixes this on my test instance by moving some of the data about in the token that the client sends. It works with 8u20 and 8u45. At the moment the hack is extremely fragile. I need to make it more robust and make it optional. I should be able to get that done tomorrow and have it included in the next Tomcat 8 release. Fix applied to trunk (for 9.0.x), 8.0.x (for 8.0.23 onwards) and 7.0.x (for 7.0.63 onwards). Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: SPNEGO test configuration with Manager webapp
On 29/03/2015 23:13, André Warnier wrote: David Marsh wrote: I've tested all the following public JDKs jdk-7u45-windows-i586.exe jdk-7u65-windows-i586.exe jdk-7u75-windows-i586.exe jdk-8-windows-i586.exe jdk-8u5-windows-i586.exe jdk-8u11-windows-i586.exe jdk-8u20-windows-i586.exe jdk-8u25-windows-i586.exe jdk-8u31-windows-i586.exe jdk-8u40-windows-i586.exe -- Only this one fails SPNEGO / Bad GSS Token Seems a recent fix must broken it. That is really great info. Thanks. As promised I have found some time to look into this. It appears that this fix in 8u40 onwards broke SPNEGO. https://bugs.openjdk.java.net/browse/JDK-8048194 The fix that was applied wasn't the one suggested in the bug report. I've spent some time looking at the code but I haven't found a way around this yet. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: SPNEGO test configuration with Manager webapp
On 14/05/2015 21:11, Mark Thomas wrote: On 29/03/2015 23:13, André Warnier wrote: David Marsh wrote: I've tested all the following public JDKs jdk-7u45-windows-i586.exe jdk-7u65-windows-i586.exe jdk-7u75-windows-i586.exe jdk-8-windows-i586.exe jdk-8u5-windows-i586.exe jdk-8u11-windows-i586.exe jdk-8u20-windows-i586.exe jdk-8u25-windows-i586.exe jdk-8u31-windows-i586.exe jdk-8u40-windows-i586.exe -- Only this one fails SPNEGO / Bad GSS Token Seems a recent fix must broken it. That is really great info. Thanks. As promised I have found some time to look into this. It appears that this fix in 8u40 onwards broke SPNEGO. https://bugs.openjdk.java.net/browse/JDK-8048194 The fix that was applied wasn't the one suggested in the bug report. I've spent some time looking at the code but I haven't found a way around this yet. Good news (sort of). I have an *extremely* dirty hack that fixes this on my test instance by moving some of the data about in the token that the client sends. It works with 8u20 and 8u45. At the moment the hack is extremely fragile. I need to make it more robust and make it optional. I should be able to get that done tomorrow and have it included in the next Tomcat 8 release. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: ant test on Tomcat 7.0.59 is failing due to exceptions
Hi Tomcat Community, I have downloaded latest certificates from http://svn.apache.org/viewvc/tomcat/tc7.0.x/trunk/test/org/apache/tomcat/util/net/ And now test are passing. Thanks Tomcat community :) Arjit Kumar On Thu, May 7, 2015 at 9:39 AM, Arjit Gupta arjitk.gu...@gmail.com wrote: Thanks Tomcat Community for the quick reply. As answer provided by Mark the test certificate used by unit tests have been expired, So please provide me any link where I could download the certificates in order to test these unit tests successful. Thanks in Advance. Arjit Kumar On Wed, May 6, 2015 at 5:33 PM, Mark Thomas ma...@apache.org wrote: On 06/05/2015 10:43, Arjit Gupta wrote: Hello Tomcat Community, I am building tomcat 7.0.59 with java7 on HP-UX(OS) itanium(IA64) processor. After building it successful I am running *ant test*. Which is failing due to exception which is as below: Testcase: testSimpleSsl took 4.085 sec Caused an ERROR sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed javax.net.ssl.SSLHandshakeException: snip/ I have manually tested the ssl functionality on Tomcat by enabling it from server.xml .It is working fine. Please suggest the plausible cause of the above exception. The test certificates used in the unit tests have expired. Is the this exception is coming due to some problem in build or some configuration issue in OS? No. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: ant test on Tomcat 7.0.59 is failing due to exceptions
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Arjit, On 5/7/15 12:09 AM, Arjit Gupta wrote: Thanks Tomcat Community for the quick reply. As answer provided by Mark the test certificate used by unit tests have been expired, So please provide me any link where I could download the certificates in order to test these unit tests successful. You should just be able to create a new certificate with the same common name (localhost?). Or, you can pick-up a new certificate from svn if the certificate has been updated. - -chris On Wed, May 6, 2015 at 5:33 PM, Mark Thomas ma...@apache.org wrote: On 06/05/2015 10:43, Arjit Gupta wrote: Hello Tomcat Community, I am building tomcat 7.0.59 with java7 on HP-UX(OS) itanium(IA64) processor. After building it successful I am running *ant test*. Which is failing due to exception which is as below: Testcase: testSimpleSsl took 4.085 sec Caused an ERROR sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed javax.net.ssl.SSLHandshakeException: snip/ I have manually tested the ssl functionality on Tomcat by enabling it from server.xml .It is working fine. Please suggest the plausible cause of the above exception. The test certificates used in the unit tests have expired. Is the this exception is coming due to some problem in build or some configuration issue in OS? No. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org -BEGIN PGP SIGNATURE- Version: GnuPG v2 Comment: GPGTools - http://gpgtools.org iQIcBAEBCAAGBQJVS1FPAAoJEBzwKT+lPKRYUqsP/2dMJSHOy/wU8oUs1R9gUyXZ vU1/zeBPWZFlvdX/tW5+mWdWuD4CadI6PaEVhvwX/hVz7+Oi0XgThlcGc+y8thDb bmECjXeWr8QRol6QQCFrYR/l51OYiICiU+ZCnKQnZ07gSTb0N+C7/+ynTCgvB79G PrrLGQXd3OnvwmiLpHmrgXeupZva7yUh46WfalffUiimDRwrwBegO6I4x0+QQINb os20rEGx7TpnCWLZIMH2SqWJgImDTrAsAvXr2fiJVXVSu8y3WvVvT4tG2w4XV+oW FhbNMjpaI8s1PjO4x8IDCVWgtn/97gtbVfn2tTpDD5V/Ym+zvMj4OnUr3Qrw8Dqn yd1xYx1/bUdh107biiaDfrvF2bP++eaGEtTabN6M9QlcqnAosd2yW0fWLISs7MtB YP3cR5GYhvx2pCTC74sLc2jClclWdUm+yPp9RR2fhthY3xfOxQ/k3+8CEXW5BOLM K/lLTgP2R1pNUArJ0M9jamj/LwVLty+TrJ+63rk6uLKMo+R5371vu1MMfDCqqSQB rxtsZnBi7nkpoNglBZPAFopneijiW0VKcL+9HKCxjclnV9eHgPBdr0sW034g8lQd qYL0J0smIef3bYbgNYcNfKngAJ9H9mOuIKUdbCM2cHPJSpjrmL4BT2OmUy8rgxn5 tbFeeaHpYk6x/bfNFmf9 =wqOA -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
ant test on Tomcat 7.0.59 is failing due to exceptions
Hello Tomcat Community, I am building tomcat 7.0.59 with java7 on HP-UX(OS) itanium(IA64) processor. After building it successful I am running *ant test*. Which is failing due to exception which is as below: Testcase: testSimpleSsl took 4.085 sec Caused an ERROR sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1886) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) at sun.security.ssl.Handshaker.process_record(Handshaker.java:804) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323) at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:515) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153) at org.apache.catalina.startup.TomcatBaseTest.methodUrl(TomcatBaseTest.java:636) at org.apache.catalina.startup.TomcatBaseTest.getUrl(TomcatBaseTest.java:609) at org.apache.catalina.startup.TomcatBaseTest.getUrl(TomcatBaseTest.java:603) at org.apache.catalina.startup.TomcatBaseTest.getUrl(TomcatBaseTest.java:592) at org.apache.catalina.startup.TomcatBaseTest.getUrl(TomcatBaseTest.java:586) at org.apache.tomcat.util.net.TestSsl.testSimpleSsl(TestSsl.java:62) Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:350) at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:260) at sun.security.validator.Validator.validate(Validator.java:260) at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231) at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:126) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1323) Caused by: java.security.cert.CertPathValidatorException: timestamp check failed at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:159) at sun.security.provider.certpath.PKIXCertPathValidator.doValidate(PKIXCertPathValidator.java:351) at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:191) at java.security.cert.CertPathValidator.validate(CertPathValidator.java:279) at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:345) Caused by: java.security.cert.CertificateExpiredException: NotAfter: Sat Feb 28 10:58:42 IST 2015 at sun.security.x509.CertificateValidity.valid(CertificateValidity.java:273) at sun.security.x509.X509CertImpl.checkValidity(X509CertImpl.java:581) at sun.security.provider.certpath.BasicChecker.verifyTimestamp(BasicChecker.java:184) at sun.security.provider.certpath.BasicChecker.check(BasicChecker.java:136) at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:133) This exception is coming from various test files as below: TestClientCert.java TestCustomSsl.java TestSsl.java I have manually tested the ssl functionality on Tomcat by enabling it from server.xml .It is working fine. Please suggest the plausible cause of the above exception. Is the this exception is coming due to some problem in build or some configuration issue in OS? Thanks in Advance. Arjit Kumar
Re: ant test on Tomcat 7.0.59 is failing due to exceptions
On 06/05/2015 10:43, Arjit Gupta wrote: Hello Tomcat Community, I am building tomcat 7.0.59 with java7 on HP-UX(OS) itanium(IA64) processor. After building it successful I am running *ant test*. Which is failing due to exception which is as below: Testcase: testSimpleSsl took 4.085 sec Caused an ERROR sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed javax.net.ssl.SSLHandshakeException: snip/ I have manually tested the ssl functionality on Tomcat by enabling it from server.xml .It is working fine. Please suggest the plausible cause of the above exception. The test certificates used in the unit tests have expired. Is the this exception is coming due to some problem in build or some configuration issue in OS? No. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: ant test on Tomcat 7.0.59 is failing due to exceptions
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Mark, On 5/6/15 8:03 AM, Mark Thomas wrote: On 06/05/2015 10:43, Arjit Gupta wrote: Hello Tomcat Community, I am building tomcat 7.0.59 with java7 on HP-UX(OS) itanium(IA64) processor. After building it successful I am running *ant test*. Which is failing due to exception which is as below: Testcase: testSimpleSsl took 4.085 sec Caused an ERROR sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed javax.net.ssl.SSLHandshakeException: snip/ I have manually tested the ssl functionality on Tomcat by enabling it from server.xml .It is working fine. Please suggest the plausible cause of the above exception. The test certificates used in the unit tests have expired. Is the this exception is coming due to some problem in build or some configuration issue in OS? No. Is there any reason not to auto-generate test certs at the beginning of the ant test target? Also, it's possible to disable certain portions of the certificate-checking algorithm. Would it be worth it to implement those workarounds so that expired certs won't fail? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v2 Comment: GPGTools - http://gpgtools.org iQIcBAEBCAAGBQJVSg6kAAoJEBzwKT+lPKRYh7gP/0UxglztKH4NT1o82KdPws8/ XDLUs6O96LTQWg2XvsXpbcBvO93nBHoStojigGV8IiDdLl2Jcju+ggH+KHIc0Ywv SjCSI9s981tk+F7sJOQibNCwxWWCaP+bG1oydPJukCsd+OB4tk6ltP8tPcASjo7u pU88bTxlJMT3T97B3QnVhZV8UOtZQQxISA5bZOU3l/UhjLsV7zfCxqu/1Bki7u3F 2+hD5iwY4qrU52+iyeqzWL3NNFsd4ygO5aWJyELD5WQS50e8otjMvBgD9gZ7DN+H nvz9eJWsrlIxyZmpjDBtxBTfPua5BJ4Ne/Im/QUe5LAYrxHlR751VhG/tVPMfJNM OvAMEqP2bmL5b0cPpcZxY8j8t1J951Wxp9EhK2UhapCMDUfrmO7xDrFH4Oc/fUjq oLS8Tgqws5BE/blhjwHgX3TxemrwzBcbSAFJoGoOo32qr9ZjWP3V5k8nU2u7YNZR nH7IKyMOTOxkPtcKG1HUaVFMPl68+0rbRnJpbQHqz8wi9WUgokuEhB9oSL1egFNs LptNDYjsfc696GZrjtQnnxvS6ViW5+q9AokaddO+xelbtTUAZ7Y3FsXK/QDJuKoX AUVz6eTTC8GLTMt1AFvM+SKL3PKIAAZHPv7u2o+hLHtViSUKBlqB5IaQs6RFiRN2 f8dqOtaSmUWHsIEFSzTJ =cxQ4 -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: ant test on Tomcat 7.0.59 is failing due to exceptions
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Mark, On 5/6/15 9:01 AM, Mark Thomas wrote: On 06/05/2015 13:52, Christopher Schultz wrote: Mark, On 5/6/15 8:03 AM, Mark Thomas wrote: On 06/05/2015 10:43, Arjit Gupta wrote: Hello Tomcat Community, I am building tomcat 7.0.59 with java7 on HP-UX(OS) itanium(IA64) processor. After building it successful I am running *ant test*. Which is failing due to exception which is as below: Testcase: testSimpleSsl took 4.085 sec Caused an ERROR sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed javax.net.ssl.SSLHandshakeException: snip/ I have manually tested the ssl functionality on Tomcat by enabling it from server.xml .It is working fine. Please suggest the plausible cause of the above exception. The test certificates used in the unit tests have expired. Is the this exception is coming due to some problem in build or some configuration issue in OS? No. Is there any reason not to auto-generate test certs at the beginning of the ant test target? In principle no. In theory, entropy issues? Good point. How many certificates do we need? Perhaps we could have an ant property that controls whether generation of a new certificate would happen or not. Also, it's possible to disable certain portions of the certificate-checking algorithm. Would it be worth it to implement those workarounds so that expired certs won't fail? I prefer the generation option. Okay. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v2 Comment: GPGTools - http://gpgtools.org iQIcBAEBCAAGBQJVShQnAAoJEBzwKT+lPKRYcoQP/17dQROT3E6hXOyiEHVesPn+ 34qF75s3HXkjgI/aR6k1+74UHWwfaIXgfkHvb0aFrIh2rcnP9YKAlzNg/wA3iADN MfInjrlwu1shEAg1FkJw8vjzfgt4xoGnxOZ/1UEDrUQEdnSiHCqGP95x0upRjKQe 62oYIe8zDSbQL0v/AGgNtr6MGaZZ/VKwd/eLDFNC3Y0uGVlmDjAFYUNSPLgGKlDQ EicBNeAh365wPHTK/9fwmcuvcuiGUcs9ruoviJOZ2jSghcbdwJE4ONhlL7xuOAGv 3LfOdHA9XtAFJMW87SeZ9LfMZjdxc04yLorr5H3pBVHEesisPwHCl9WOL/yvkfEM st8lkQpCklykFpryx2e92s/c7K+2nK+ym5RcjU7cao1ImoDSU1w0cnvUctiJDqOf Tlh2+6IgOzuIF5BLMbK4bkk4UnLivzRy9XwEOBrfgkzenb1M2Sg/xgrBHHQLnpHr xeKYJTO7dgxrBZ2EYe9NTNoNqvUztt+bywk/uVcDbaVnjb0Kt0epaQJp5Cb5gFbS VHjXyawirg3FWQTFPl3MIq8cMmJBDSjsuN4O1MFfxXx7rrS74MGxTz1s/UjETU2C diC3iG/WpzI/ODkPXH00Cv0/bxrvkAL25YjShprhTFcOFhcA80ma53rlJtAlUeGV FYxV/5/hEMvxR10nfKJc =JiJE -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: ant test on Tomcat 7.0.59 is failing due to exceptions
On 06/05/2015 13:52, Christopher Schultz wrote: Mark, On 5/6/15 8:03 AM, Mark Thomas wrote: On 06/05/2015 10:43, Arjit Gupta wrote: Hello Tomcat Community, I am building tomcat 7.0.59 with java7 on HP-UX(OS) itanium(IA64) processor. After building it successful I am running *ant test*. Which is failing due to exception which is as below: Testcase: testSimpleSsl took 4.085 sec Caused an ERROR sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed javax.net.ssl.SSLHandshakeException: snip/ I have manually tested the ssl functionality on Tomcat by enabling it from server.xml .It is working fine. Please suggest the plausible cause of the above exception. The test certificates used in the unit tests have expired. Is the this exception is coming due to some problem in build or some configuration issue in OS? No. Is there any reason not to auto-generate test certs at the beginning of the ant test target? In principle no. In theory, entropy issues? Also, it's possible to disable certain portions of the certificate-checking algorithm. Would it be worth it to implement those workarounds so that expired certs won't fail? I prefer the generation option. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: ant test on Tomcat 7.0.59 is failing due to exceptions
2015-05-06 15:52 GMT+03:00 Christopher Schultz Mark, On 5/6/15 8:03 AM, Mark Thomas wrote: On 06/05/2015 10:43, Arjit Gupta wrote: Hello Tomcat Community, I am building tomcat 7.0.59 with java7 on HP-UX(OS) itanium(IA64) processor. After building it successful I am running *ant test*. Which is failing due to exception which is as below: Testcase: testSimpleSsl took 4.085 sec Caused an ERROR sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed javax.net.ssl.SSLHandshakeException: snip/ I have manually tested the ssl functionality on Tomcat by enabling it from server.xml .It is working fine. Please suggest the plausible cause of the above exception. The test certificates used in the unit tests have expired. Is the this exception is coming due to some problem in build or some configuration issue in OS? No. Is there any reason not to auto-generate test certs at the beginning of the ant test target? Also, it's possible to disable certain portions of the certificate-checking algorithm. Would it be worth it to implement those workarounds so that expired certs won't fail? 1) Document this issue in BUILDING.txt 2) Usually people can get recent certificates from our source tree in svn and swap them in. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: ant test on Tomcat 7.0.59 is failing due to exceptions
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Konstantin, On 5/6/15 9:25 AM, Konstantin Kolinko wrote: 2015-05-06 15:52 GMT+03:00 Christopher Schultz Mark, On 5/6/15 8:03 AM, Mark Thomas wrote: On 06/05/2015 10:43, Arjit Gupta wrote: Hello Tomcat Community, I am building tomcat 7.0.59 with java7 on HP-UX(OS) itanium(IA64) processor. After building it successful I am running *ant test*. Which is failing due to exception which is as below: Testcase: testSimpleSsl took 4.085 sec Caused an ERROR sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed javax.net.ssl.SSLHandshakeException: snip/ I have manually tested the ssl functionality on Tomcat by enabling it from server.xml .It is working fine. Please suggest the plausible cause of the above exception. The test certificates used in the unit tests have expired. Is the this exception is coming due to some problem in build or some configuration issue in OS? No. Is there any reason not to auto-generate test certs at the beginning of the ant test target? Also, it's possible to disable certain portions of the certificate-checking algorithm. Would it be worth it to implement those workarounds so that expired certs won't fail? 1) Document this issue in BUILDING.txt 2) Usually people can get recent certificates from our source tree in svn and swap them in. So in order to successfully run unit tests against an old package (e.g. Tomcat 7.0.5), one has to fetch artifacts from the current source tree? That's just ugly. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v2 Comment: GPGTools - http://gpgtools.org iQIcBAEBCAAGBQJVShc5AAoJEBzwKT+lPKRY5YAQAMCkycg/Y30ZJyNbYhlrvZUR YGTszCKEsS9g0VZTpSD6Mlxu5PviLYf1o0ABADl/HyLfi57VpCO+u1wrwUoIYEYa BeDCFonBW2rURjFAejy2cxgjGJetnfSdaSk5el3BznZs8ThmfZuiHIsfNYbYMS0e RsGECuW/DK92A20UYa1vwD8TQcWtXxTYnWpq/XWUwNwjvK9vmboZ5n7UODb/he1Q tQiHpw6hnsAdH2ckZ98IP0KjfuvlFn7BQtSRBF+GcFL5dAcDvET8cmYJI0ENjFav M+Aom5MHzQBR4GVlVqxctL/KtPm5pHP9hWiraDUXiW9BoprnnKyG+de9Enwerlfd CI3y/b5qX2JypwVxfBFToSUOon700+/Av5sT6VEB2OYAIIb+ukqFAQpnUE+AkaID +otfDy56DZM44haffG0VLAnt1h0frGSx4kmyCT0BjO3fm6Fgelrd1VT0Ys3A53JS D4Sc5rwI/Ws4c2GaXHbaVM5c2MGf0AUVZXbf++Wv7O1PAf9wjKu9Xy6MXFIXJkuM nBYN8Ydzve9LIQM+h5kH9W/hwNMephFmpsquhQXazYADatekaYlMPlmZgEoqIZQb 5VE60u72DXLdVZ3a+AQCb0pgHLWfdG9xzQ+m8kRmnQA3Vfc5OlALU2ZOch6vD3za jeiuM5AIgEFJUk63glsa =4xIj -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: ant test on Tomcat 7.0.59 is failing due to exceptions
Thanks Tomcat Community for the quick reply. As answer provided by Mark the test certificate used by unit tests have been expired, So please provide me any link where I could download the certificates in order to test these unit tests successful. Thanks in Advance. Arjit Kumar On Wed, May 6, 2015 at 5:33 PM, Mark Thomas ma...@apache.org wrote: On 06/05/2015 10:43, Arjit Gupta wrote: Hello Tomcat Community, I am building tomcat 7.0.59 with java7 on HP-UX(OS) itanium(IA64) processor. After building it successful I am running *ant test*. Which is failing due to exception which is as below: Testcase: testSimpleSsl took 4.085 sec Caused an ERROR sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed javax.net.ssl.SSLHandshakeException: snip/ I have manually tested the ssl functionality on Tomcat by enabling it from server.xml .It is working fine. Please suggest the plausible cause of the above exception. The test certificates used in the unit tests have expired. Is the this exception is coming due to some problem in build or some configuration issue in OS? No. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: SPNEGO test configuration with Manager webapp
Am 28. März 2015 17:46:50 MEZ, schrieb Mark Thomas ma...@apache.org: On 28/03/2015 14:43, David Marsh wrote: Ok so I went back to basics and created three new VM's. Windows Server 2008 R2 Windows 7 Client Windows 7 Tomcat I still had same issues, until I changed the Java on the tomcat server to JDK 7 u45. It appears there are breaking changes to JAAS/GSS in newer JDKs ? Thank you for doing all this testing. That is useful information to know. The next step (for you, me or anyone who has the time and wants to help) is to test subsequent Java 7 releases and see at which version it stops working. I'd hope that a review of the relevant change log would identify the change that triggered the breakage and provide some clues on how to fix it. It would be worth testing the Java 8 releases the same way. I read it, that jdk 7 works and jdk 8 is problematic. There are a few Kerberos related Chaves in jdk 8 ( http://docs.oracle.com/javase/8/docs/technotes/guides/security/enhancements-8.html). Interesting are the two changes: * DES is disabled by default * constrained delegation is supported. My guess would be, that it would help (in this case) to reenable DES by adding allow_weak_crypto=true in the krb5.conf. Regards Felix Mark David From: dmars...@outlook.com To: users@tomcat.apache.org Subject: RE: SPNEGO test configuration with Manager webapp Date: Fri, 27 Mar 2015 23:40:06 + By the way Tomcat 8 was running on JDK :- C:\Windows\system32java -version java version 1.8.0_40 Java(TM) SE Runtime Environment (build 1.8.0_40-b26) Java HotSpot(TM) Client VM (build 25.40-b25, mixed mode) Version update 40 should include some JRE fixes around GSS and SPNEGO, including ignoring parts of NegoEx, however it does not seem to work. I've also created a Windows 7 client with same config just different DNS of win-pc02.kerbtest.local It has the same issue going from firefox to http://win-tc01.kerbtest.local/manager/html I get the same three 401's and the Negotiate. Date: Thu, 26 Mar 2015 12:11:34 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks for that, yes I've got 30 years windows experience, I can use Linux at a push but its not really my area expertise. I'm a Java / Windows programmer so I should be able to understand it, but not kerberos or Active Directory expert. I have used Waffle in the past with success and used JAAS/GSS-API in Java thick clients. I made the IE settings you outlined but it seems to still prompt. IE has win-tc01.kerbtest.local as a trusted site. Enable Windows Integrated Authentication is on Auto logon only in Intranet Zone is on I've been using Firefox to test and that does send 401 and negotiate, but causes the GSS token error mentioned. Active directory and krb5.ini are using eType 23 which is rc4-hmac The windows client OS and tomcat server OS has registry setting for allowtgtsessionkey set to 1 (enabled). Java kinit test works and stores a ticket in the Java session cache. So problem seems to be either :- 1. Browser sends bad token 2. Token is good but Oracle JDK 8 GSS-API cannot handle it Another shot almost in the dark : while browsing hundreds of Kerberos-related pages on the WWW, one other recommendation which seems to appear regularly (and Mark also mentioned that somewhere), is that each time you make a change somewhere, you should reboot the machine afterward, before re-testing. (Particularly on Windows machines). I know it's a PITA, but I have also found the same to be true sometimes when merely dealing with NTLM matters. There are probably some hidden caches that get cleared only in that way. many thanks David Date: Thu, 26 Mar 2015 11:32:39 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks that would be great ! Do you have a good mechanism to test and ensure kerberos token is passed to tomcat and not NTLM token ? I believe that I can answer that. And the basic answer is no. First the basic principle, valid for this and many many other areas : the server cannot impose anything on the browser. The local user can always override anything received from the server, by a setting in the browser. And a hacker can of course do anything. All the server can do, is tell the browser what it will accept, and the browser can tell the server ditto. So, never assume the opposite, and you will save yourself a lot of fruitless searches and dead-ends. Now more specific : 1) For Kerberos to be used at all at the browser level, the server must send a 401 response with Negociate as the requested authentication method. Unless it does that, the browser will never even attempt to send a Kerberos Authorization
RE: SPNEGO test configuration with Manager webapp
I've tested all the following public JDKs jdk-7u45-windows-i586.exe jdk-7u65-windows-i586.exe jdk-7u75-windows-i586.exe jdk-8-windows-i586.exe jdk-8u5-windows-i586.exe jdk-8u11-windows-i586.exe jdk-8u20-windows-i586.exe jdk-8u25-windows-i586.exe jdk-8u31-windows-i586.exe jdk-8u40-windows-i586.exe -- Only this one fails SPNEGO / Bad GSS Token Seems a recent fix must broken it. David Subject: Re: SPNEGO test configuration with Manager webapp From: felix.schumac...@internetallee.de Date: Sun, 29 Mar 2015 10:13:29 +0200 To: users@tomcat.apache.org Am 28. März 2015 17:46:50 MEZ, schrieb Mark Thomas ma...@apache.org: On 28/03/2015 14:43, David Marsh wrote: Ok so I went back to basics and created three new VM's. Windows Server 2008 R2 Windows 7 Client Windows 7 Tomcat I still had same issues, until I changed the Java on the tomcat server to JDK 7 u45. It appears there are breaking changes to JAAS/GSS in newer JDKs ? Thank you for doing all this testing. That is useful information to know. The next step (for you, me or anyone who has the time and wants to help) is to test subsequent Java 7 releases and see at which version it stops working. I'd hope that a review of the relevant change log would identify the change that triggered the breakage and provide some clues on how to fix it. It would be worth testing the Java 8 releases the same way. I read it, that jdk 7 works and jdk 8 is problematic. There are a few Kerberos related Chaves in jdk 8 ( http://docs.oracle.com/javase/8/docs/technotes/guides/security/enhancements-8.html). Interesting are the two changes: * DES is disabled by default * constrained delegation is supported. My guess would be, that it would help (in this case) to reenable DES by adding allow_weak_crypto=true in the krb5.conf. Regards Felix Mark David From: dmars...@outlook.com To: users@tomcat.apache.org Subject: RE: SPNEGO test configuration with Manager webapp Date: Fri, 27 Mar 2015 23:40:06 + By the way Tomcat 8 was running on JDK :- C:\Windows\system32java -version java version 1.8.0_40 Java(TM) SE Runtime Environment (build 1.8.0_40-b26) Java HotSpot(TM) Client VM (build 25.40-b25, mixed mode) Version update 40 should include some JRE fixes around GSS and SPNEGO, including ignoring parts of NegoEx, however it does not seem to work. I've also created a Windows 7 client with same config just different DNS of win-pc02.kerbtest.local It has the same issue going from firefox to http://win-tc01.kerbtest.local/manager/html I get the same three 401's and the Negotiate. Date: Thu, 26 Mar 2015 12:11:34 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks for that, yes I've got 30 years windows experience, I can use Linux at a push but its not really my area expertise. I'm a Java / Windows programmer so I should be able to understand it, but not kerberos or Active Directory expert. I have used Waffle in the past with success and used JAAS/GSS-API in Java thick clients. I made the IE settings you outlined but it seems to still prompt. IE has win-tc01.kerbtest.local as a trusted site. Enable Windows Integrated Authentication is on Auto logon only in Intranet Zone is on I've been using Firefox to test and that does send 401 and negotiate, but causes the GSS token error mentioned. Active directory and krb5.ini are using eType 23 which is rc4-hmac The windows client OS and tomcat server OS has registry setting for allowtgtsessionkey set to 1 (enabled). Java kinit test works and stores a ticket in the Java session cache. So problem seems to be either :- 1. Browser sends bad token 2. Token is good but Oracle JDK 8 GSS-API cannot handle it Another shot almost in the dark : while browsing hundreds of Kerberos-related pages on the WWW, one other recommendation which seems to appear regularly (and Mark also mentioned that somewhere), is that each time you make a change somewhere, you should reboot the machine afterward, before re-testing. (Particularly on Windows machines). I know it's a PITA, but I have also found the same to be true sometimes when merely dealing with NTLM matters. There are probably some hidden caches that get cleared only in that way. many thanks David Date: Thu, 26 Mar 2015 11:32:39 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks that would be great ! Do you have a good mechanism to test and ensure kerberos token is passed to tomcat and not NTLM token ? I believe that I can answer that. And the basic answer is no. First the basic principle, valid for this and many many other areas : the server cannot impose anything on the browser. The local
Re: SPNEGO test configuration with Manager webapp
David Marsh wrote: I've tested all the following public JDKs jdk-7u45-windows-i586.exe jdk-7u65-windows-i586.exe jdk-7u75-windows-i586.exe jdk-8-windows-i586.exe jdk-8u5-windows-i586.exe jdk-8u11-windows-i586.exe jdk-8u20-windows-i586.exe jdk-8u25-windows-i586.exe jdk-8u31-windows-i586.exe jdk-8u40-windows-i586.exe -- Only this one fails SPNEGO / Bad GSS Token Seems a recent fix must broken it. That is really great info. Thanks. By the way, would you still have the Tomcat Kerberos logs that fail, in comparison to one where it works ? David Subject: Re: SPNEGO test configuration with Manager webapp From: felix.schumac...@internetallee.de Date: Sun, 29 Mar 2015 10:13:29 +0200 To: users@tomcat.apache.org Am 28. März 2015 17:46:50 MEZ, schrieb Mark Thomas ma...@apache.org: On 28/03/2015 14:43, David Marsh wrote: Ok so I went back to basics and created three new VM's. Windows Server 2008 R2 Windows 7 Client Windows 7 Tomcat I still had same issues, until I changed the Java on the tomcat server to JDK 7 u45. It appears there are breaking changes to JAAS/GSS in newer JDKs ? Thank you for doing all this testing. That is useful information to know. The next step (for you, me or anyone who has the time and wants to help) is to test subsequent Java 7 releases and see at which version it stops working. I'd hope that a review of the relevant change log would identify the change that triggered the breakage and provide some clues on how to fix it. It would be worth testing the Java 8 releases the same way. I read it, that jdk 7 works and jdk 8 is problematic. There are a few Kerberos related Chaves in jdk 8 ( http://docs.oracle.com/javase/8/docs/technotes/guides/security/enhancements-8.html). Interesting are the two changes: * DES is disabled by default * constrained delegation is supported. My guess would be, that it would help (in this case) to reenable DES by adding allow_weak_crypto=true in the krb5.conf. Regards Felix Mark David From: dmars...@outlook.com To: users@tomcat.apache.org Subject: RE: SPNEGO test configuration with Manager webapp Date: Fri, 27 Mar 2015 23:40:06 + By the way Tomcat 8 was running on JDK :- C:\Windows\system32java -version java version 1.8.0_40 Java(TM) SE Runtime Environment (build 1.8.0_40-b26) Java HotSpot(TM) Client VM (build 25.40-b25, mixed mode) Version update 40 should include some JRE fixes around GSS and SPNEGO, including ignoring parts of NegoEx, however it does not seem to work. I've also created a Windows 7 client with same config just different DNS of win-pc02.kerbtest.local It has the same issue going from firefox to http://win-tc01.kerbtest.local/manager/html I get the same three 401's and the Negotiate. Date: Thu, 26 Mar 2015 12:11:34 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks for that, yes I've got 30 years windows experience, I can use Linux at a push but its not really my area expertise. I'm a Java / Windows programmer so I should be able to understand it, but not kerberos or Active Directory expert. I have used Waffle in the past with success and used JAAS/GSS-API in Java thick clients. I made the IE settings you outlined but it seems to still prompt. IE has win-tc01.kerbtest.local as a trusted site. Enable Windows Integrated Authentication is on Auto logon only in Intranet Zone is on I've been using Firefox to test and that does send 401 and negotiate, but causes the GSS token error mentioned. Active directory and krb5.ini are using eType 23 which is rc4-hmac The windows client OS and tomcat server OS has registry setting for allowtgtsessionkey set to 1 (enabled). Java kinit test works and stores a ticket in the Java session cache. So problem seems to be either :- 1. Browser sends bad token 2. Token is good but Oracle JDK 8 GSS-API cannot handle it Another shot almost in the dark : while browsing hundreds of Kerberos-related pages on the WWW, one other recommendation which seems to appear regularly (and Mark also mentioned that somewhere), is that each time you make a change somewhere, you should reboot the machine afterward, before re-testing. (Particularly on Windows machines). I know it's a PITA, but I have also found the same to be true sometimes when merely dealing with NTLM matters. There are probably some hidden caches that get cleared only in that way. many thanks David Date: Thu, 26 Mar 2015 11:32:39 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks that would be great ! Do you have a good mechanism to test and ensure kerberos token is passed to tomcat and not NTLM token ? I believe that I can answer that. And the basic answer
RE: SPNEGO test configuration with Manager webapp
Broken trace :- 25-Mar-2015 15:46:22.131 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service Catalina 25-Mar-2015 15:46:22.133 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.0.20 25-Mar-2015 15:46:22.257 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployD irectory Deploying web application directory C:\Program Files\Apache Software Foundation\Tomcat 8.0\ webapps\docs 25-Mar-2015 15:46:22.637 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployD irectory Deployment of web application directory C:\Program Files\Apache Software Foundation\Tomcat 8.0\webapps\docs has finished in 380 ms 25-Mar-2015 15:46:22.639 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployD irectory Deploying web application directory C:\Program Files\Apache Software Foundation\Tomcat 8.0\ webapps\manager 25-Mar-2015 15:46:22.710 FINE [localhost-startStop-1] org.apache.catalina.authenticator.Authenticato rBase.startInternal No SingleSignOn Valve is present 25-Mar-2015 15:46:22.733 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployD irectory Deployment of web application directory C:\Program Files\Apache Software Foundation\Tomcat 8.0\webapps\manager has finished in 93 ms 25-Mar-2015 15:46:22.734 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployD irectory Deploying web application directory C:\Program Files\Apache Software Foundation\Tomcat 8.0\ webapps\ROOT 25-Mar-2015 15:46:22.793 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployD irectory Deployment of web application directory C:\Program Files\Apache Software Foundation\Tomcat 8.0\webapps\ROOT has finished in 59 ms 25-Mar-2015 15:46:22.797 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandl er [http-nio-80] 25-Mar-2015 15:46:22.806 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandl er [ajp-nio-8009] 25-Mar-2015 15:46:22.808 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 72 1 ms 25-Mar-2015 15:46:28.280 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Security checking request GET /manager/html 25-Mar-2015 15:46:28.284 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Status interface]' against GET /html -- false 25-Mar-2015 15:46:28.286 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[JMX Proxy interface]' against GET /html -- fal se 25-Mar-2015 15:46:28.287 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Text Manager interface (for scripts)]' against GET /html -- false 25-Mar-2015 15:46:28.288 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[HTML Manager interface (for humans)]' against G ET /html -- true 25-Mar-2015 15:46:28.290 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Status interface]' against GET /html -- false 25-Mar-2015 15:46:28.291 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[JMX Proxy interface]' against GET /html -- fal se 25-Mar-2015 15:46:28.291 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Text Manager interface (for scripts)]' against GET /html -- false 25-Mar-2015 15:46:28.293 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[HTML Manager interface (for humans)]' against G ET /html -- true 25-Mar-2015 15:46:28.296 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Calling hasUserDataPermission() 25-Mar-2015 15:46:28.299 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.hasUserDataPe rmission User data constraint has no restrictions 25-Mar-2015 15:46:28.302 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Calling authenticate() 25-Mar-2015 15:46:28.304 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.SpnegoAuthentic ator.authenticate No authorization header sent by client 25-Mar-2015 15:46:28.305 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Failed authenticate() test 25-Mar-2015 15:46:28.417 FINE [http-nio-80-exec-2] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Security checking request GET /manager/html 25-Mar-2015 15:46:28.420 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints
RE: SPNEGO test configuration with Manager webapp
[localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployD irectory Deployment of web application directory C:\Program Files (x86)\Apache Software Foundation\T omcat 8.0\webapps\docs has finished in 3,916 ms 28-Mar-2015 14:20:38.335 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployD irectory Deploying web application directory C:\Program Files (x86)\Apache Software Foundation\Tomca t 8.0\webapps\manager 28-Mar-2015 14:20:38.585 FINE [localhost-startStop-1] org.apache.catalina.authenticator.Authenticato rBase.startInternal No SingleSignOn Valve is present 28-Mar-2015 14:20:38.772 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployD irectory Deployment of web application directory C:\Program Files (x86)\Apache Software Foundation\T omcat 8.0\webapps\manager has finished in 437 ms 28-Mar-2015 14:20:38.788 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployD irectory Deploying web application directory C:\Program Files (x86)\Apache Software Foundation\Tomca t 8.0\webapps\ROOT 28-Mar-2015 14:20:39.006 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployD irectory Deployment of web application directory C:\Program Files (x86)\Apache Software Foundation\T omcat 8.0\webapps\ROOT has finished in 218 ms 28-Mar-2015 14:20:39.037 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandl er [http-nio-80] 28-Mar-2015 14:20:39.084 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandl er [ajp-nio-8009] 28-Mar-2015 14:20:39.115 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 65 24 ms 28-Mar-2015 14:21:03.119 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Security checking request GET /manager/html 28-Mar-2015 14:21:23.809 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[HTML Manager interface (for humans)]' against G ET /html -- true 28-Mar-2015 14:21:23.809 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Status interface]' against GET /html -- false 28-Mar-2015 14:21:23.824 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Text Manager interface (for scripts)]' against GET /html -- false 28-Mar-2015 14:21:23.840 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[JMX Proxy interface]' against GET /html -- fal se 28-Mar-2015 14:21:23.855 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[HTML Manager interface (for humans)]' against G ET /html -- true 28-Mar-2015 14:21:23.871 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Status interface]' against GET /html -- false 28-Mar-2015 14:21:23.887 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Text Manager interface (for scripts)]' against GET /html -- false 28-Mar-2015 14:21:23.887 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[JMX Proxy interface]' against GET /html -- fal se 28-Mar-2015 14:21:23.918 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Calling hasUserDataPermission() 28-Mar-2015 14:21:23.918 FINE [http-nio-80-exec-1] org.apache.catalina.realm.RealmBase.hasUserDataPe rmission User data constraint has no restrictions 28-Mar-2015 14:21:23.933 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Calling authenticate() 28-Mar-2015 14:21:23.933 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.SpnegoAuthentic ator.authenticate No authorization header sent by client 28-Mar-2015 14:21:23.949 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Failed authenticate() test 28-Mar-2015 14:21:24.433 FINE [http-nio-80-exec-2] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Security checking request GET /manager/html 28-Mar-2015 14:21:24.448 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[HTML Manager interface (for humans)]' against G ET /html -- true 28-Mar-2015 14:21:24.464 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Status interface]' against GET /html -- false 28-Mar-2015 14:21:24.479 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint
RE: SPNEGO test configuration with Manager webapp
Ok so I went back to basics and created three new VM's. Windows Server 2008 R2 Windows 7 Client Windows 7 Tomcat I still had same issues, until I changed the Java on the tomcat server to JDK 7 u45. It appears there are breaking changes to JAAS/GSS in newer JDKs ? David From: dmars...@outlook.com To: users@tomcat.apache.org Subject: RE: SPNEGO test configuration with Manager webapp Date: Fri, 27 Mar 2015 23:40:06 + By the way Tomcat 8 was running on JDK :- C:\Windows\system32java -version java version 1.8.0_40 Java(TM) SE Runtime Environment (build 1.8.0_40-b26) Java HotSpot(TM) Client VM (build 25.40-b25, mixed mode) Version update 40 should include some JRE fixes around GSS and SPNEGO, including ignoring parts of NegoEx, however it does not seem to work. I've also created a Windows 7 client with same config just different DNS of win-pc02.kerbtest.local It has the same issue going from firefox to http://win-tc01.kerbtest.local/manager/html I get the same three 401's and the Negotiate. Date: Thu, 26 Mar 2015 12:11:34 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks for that, yes I've got 30 years windows experience, I can use Linux at a push but its not really my area expertise. I'm a Java / Windows programmer so I should be able to understand it, but not kerberos or Active Directory expert. I have used Waffle in the past with success and used JAAS/GSS-API in Java thick clients. I made the IE settings you outlined but it seems to still prompt. IE has win-tc01.kerbtest.local as a trusted site. Enable Windows Integrated Authentication is on Auto logon only in Intranet Zone is on I've been using Firefox to test and that does send 401 and negotiate, but causes the GSS token error mentioned. Active directory and krb5.ini are using eType 23 which is rc4-hmac The windows client OS and tomcat server OS has registry setting for allowtgtsessionkey set to 1 (enabled). Java kinit test works and stores a ticket in the Java session cache. So problem seems to be either :- 1. Browser sends bad token 2. Token is good but Oracle JDK 8 GSS-API cannot handle it Another shot almost in the dark : while browsing hundreds of Kerberos-related pages on the WWW, one other recommendation which seems to appear regularly (and Mark also mentioned that somewhere), is that each time you make a change somewhere, you should reboot the machine afterward, before re-testing. (Particularly on Windows machines). I know it's a PITA, but I have also found the same to be true sometimes when merely dealing with NTLM matters. There are probably some hidden caches that get cleared only in that way. many thanks David Date: Thu, 26 Mar 2015 11:32:39 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks that would be great ! Do you have a good mechanism to test and ensure kerberos token is passed to tomcat and not NTLM token ? I believe that I can answer that. And the basic answer is no. First the basic principle, valid for this and many many other areas : the server cannot impose anything on the browser. The local user can always override anything received from the server, by a setting in the browser. And a hacker can of course do anything. All the server can do, is tell the browser what it will accept, and the browser can tell the server ditto. So, never assume the opposite, and you will save yourself a lot of fruitless searches and dead-ends. Now more specific : 1) For Kerberos to be used at all at the browser level, the server must send a 401 response with Negociate as the requested authentication method. Unless it does that, the browser will never even attempt to send a Kerberos Authorization back. 2) for the browser to consider returning a Kerberos Authorization header to the server, additional conditions depend on the browser. For IE : a) the enable Windows Integrated Authentication setting must be on (checked), whether this is done locally by the user, or part of the standard IE settings company-wide, or imposed by some network policy at corporate level. b) the server to which the browser is talking, must be known to IE as either - part of the Intranet - or at least a trusted server That is defined in IE's security zones (which again can be local, or corporation-wide). If condition (a) is not met, when the server sends a 401 Negociate, IE will fall back to NTLM, always. And there is nothing you can do about that at the server level. (Funnily enough, disabling the enable Windows Integrated Authentication at the IE level, has the effect of disabling Kerberos, but not NTLM). If condition (b
Re: SPNEGO test configuration with Manager webapp
On 28/03/2015 14:43, David Marsh wrote: Ok so I went back to basics and created three new VM's. Windows Server 2008 R2 Windows 7 Client Windows 7 Tomcat I still had same issues, until I changed the Java on the tomcat server to JDK 7 u45. It appears there are breaking changes to JAAS/GSS in newer JDKs ? Thank you for doing all this testing. That is useful information to know. The next step (for you, me or anyone who has the time and wants to help) is to test subsequent Java 7 releases and see at which version it stops working. I'd hope that a review of the relevant change log would identify the change that triggered the breakage and provide some clues on how to fix it. It would be worth testing the Java 8 releases the same way. Mark David From: dmars...@outlook.com To: users@tomcat.apache.org Subject: RE: SPNEGO test configuration with Manager webapp Date: Fri, 27 Mar 2015 23:40:06 + By the way Tomcat 8 was running on JDK :- C:\Windows\system32java -version java version 1.8.0_40 Java(TM) SE Runtime Environment (build 1.8.0_40-b26) Java HotSpot(TM) Client VM (build 25.40-b25, mixed mode) Version update 40 should include some JRE fixes around GSS and SPNEGO, including ignoring parts of NegoEx, however it does not seem to work. I've also created a Windows 7 client with same config just different DNS of win-pc02.kerbtest.local It has the same issue going from firefox to http://win-tc01.kerbtest.local/manager/html I get the same three 401's and the Negotiate. Date: Thu, 26 Mar 2015 12:11:34 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks for that, yes I've got 30 years windows experience, I can use Linux at a push but its not really my area expertise. I'm a Java / Windows programmer so I should be able to understand it, but not kerberos or Active Directory expert. I have used Waffle in the past with success and used JAAS/GSS-API in Java thick clients. I made the IE settings you outlined but it seems to still prompt. IE has win-tc01.kerbtest.local as a trusted site. Enable Windows Integrated Authentication is on Auto logon only in Intranet Zone is on I've been using Firefox to test and that does send 401 and negotiate, but causes the GSS token error mentioned. Active directory and krb5.ini are using eType 23 which is rc4-hmac The windows client OS and tomcat server OS has registry setting for allowtgtsessionkey set to 1 (enabled). Java kinit test works and stores a ticket in the Java session cache. So problem seems to be either :- 1. Browser sends bad token 2. Token is good but Oracle JDK 8 GSS-API cannot handle it Another shot almost in the dark : while browsing hundreds of Kerberos-related pages on the WWW, one other recommendation which seems to appear regularly (and Mark also mentioned that somewhere), is that each time you make a change somewhere, you should reboot the machine afterward, before re-testing. (Particularly on Windows machines). I know it's a PITA, but I have also found the same to be true sometimes when merely dealing with NTLM matters. There are probably some hidden caches that get cleared only in that way. many thanks David Date: Thu, 26 Mar 2015 11:32:39 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks that would be great ! Do you have a good mechanism to test and ensure kerberos token is passed to tomcat and not NTLM token ? I believe that I can answer that. And the basic answer is no. First the basic principle, valid for this and many many other areas : the server cannot impose anything on the browser. The local user can always override anything received from the server, by a setting in the browser. And a hacker can of course do anything. All the server can do, is tell the browser what it will accept, and the browser can tell the server ditto. So, never assume the opposite, and you will save yourself a lot of fruitless searches and dead-ends. Now more specific : 1) For Kerberos to be used at all at the browser level, the server must send a 401 response with Negociate as the requested authentication method. Unless it does that, the browser will never even attempt to send a Kerberos Authorization back. 2) for the browser to consider returning a Kerberos Authorization header to the server, additional conditions depend on the browser. For IE : a) the enable Windows Integrated Authentication setting must be on (checked), whether this is done locally by the user, or part of the standard IE settings company-wide, or imposed by some network policy at corporate level. b) the server to which the browser is talking, must be known to IE
RE: SPNEGO test configuration with Manager webapp
By the way Tomcat 8 was running on JDK :- C:\Windows\system32java -version java version 1.8.0_40 Java(TM) SE Runtime Environment (build 1.8.0_40-b26) Java HotSpot(TM) Client VM (build 25.40-b25, mixed mode) Version update 40 should include some JRE fixes around GSS and SPNEGO, including ignoring parts of NegoEx, however it does not seem to work. I've also created a Windows 7 client with same config just different DNS of win-pc02.kerbtest.local It has the same issue going from firefox to http://win-tc01.kerbtest.local/manager/html I get the same three 401's and the Negotiate. Date: Thu, 26 Mar 2015 12:11:34 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks for that, yes I've got 30 years windows experience, I can use Linux at a push but its not really my area expertise. I'm a Java / Windows programmer so I should be able to understand it, but not kerberos or Active Directory expert. I have used Waffle in the past with success and used JAAS/GSS-API in Java thick clients. I made the IE settings you outlined but it seems to still prompt. IE has win-tc01.kerbtest.local as a trusted site. Enable Windows Integrated Authentication is on Auto logon only in Intranet Zone is on I've been using Firefox to test and that does send 401 and negotiate, but causes the GSS token error mentioned. Active directory and krb5.ini are using eType 23 which is rc4-hmac The windows client OS and tomcat server OS has registry setting for allowtgtsessionkey set to 1 (enabled). Java kinit test works and stores a ticket in the Java session cache. So problem seems to be either :- 1. Browser sends bad token 2. Token is good but Oracle JDK 8 GSS-API cannot handle it Another shot almost in the dark : while browsing hundreds of Kerberos-related pages on the WWW, one other recommendation which seems to appear regularly (and Mark also mentioned that somewhere), is that each time you make a change somewhere, you should reboot the machine afterward, before re-testing. (Particularly on Windows machines). I know it's a PITA, but I have also found the same to be true sometimes when merely dealing with NTLM matters. There are probably some hidden caches that get cleared only in that way. many thanks David Date: Thu, 26 Mar 2015 11:32:39 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks that would be great ! Do you have a good mechanism to test and ensure kerberos token is passed to tomcat and not NTLM token ? I believe that I can answer that. And the basic answer is no. First the basic principle, valid for this and many many other areas : the server cannot impose anything on the browser. The local user can always override anything received from the server, by a setting in the browser. And a hacker can of course do anything. All the server can do, is tell the browser what it will accept, and the browser can tell the server ditto. So, never assume the opposite, and you will save yourself a lot of fruitless searches and dead-ends. Now more specific : 1) For Kerberos to be used at all at the browser level, the server must send a 401 response with Negociate as the requested authentication method. Unless it does that, the browser will never even attempt to send a Kerberos Authorization back. 2) for the browser to consider returning a Kerberos Authorization header to the server, additional conditions depend on the browser. For IE : a) the enable Windows Integrated Authentication setting must be on (checked), whether this is done locally by the user, or part of the standard IE settings company-wide, or imposed by some network policy at corporate level. b) the server to which the browser is talking, must be known to IE as either - part of the Intranet - or at least a trusted server That is defined in IE's security zones (which again can be local, or corporation-wide). If condition (a) is not met, when the server sends a 401 Negociate, IE will fall back to NTLM, always. And there is nothing you can do about that at the server level. (Funnily enough, disabling the enable Windows Integrated Authentication at the IE level, has the effect of disabling Kerberos, but not NTLM). If condition (b) is not met, IE will try neither Kerberos nor NTLM, and it /might/ fall back to Basic authentication, if its other settings allow that. That's when you see the browser popup login dialog; and in an SSO context, this is a sure sign that something isn't working as expected. Some authentication modules, at the server level, are able to adapt to what the browser sends, others not. I believe that Waffle can accept either browser NTLM or Kerberos authentication. Waffle works only
Re: SPNEGO test configuration with Manager webapp
David Marsh wrote: Hi Mark, Thanks that would be great ! Do you have a good mechanism to test and ensure kerberos token is passed to tomcat and not NTLM token ? I believe that I can answer that. And the basic answer is no. First the basic principle, valid for this and many many other areas : the server cannot impose anything on the browser. The local user can always override anything received from the server, by a setting in the browser. And a hacker can of course do anything. All the server can do, is tell the browser what it will accept, and the browser can tell the server ditto. So, never assume the opposite, and you will save yourself a lot of fruitless searches and dead-ends. Now more specific : 1) For Kerberos to be used at all at the browser level, the server must send a 401 response with Negociate as the requested authentication method. Unless it does that, the browser will never even attempt to send a Kerberos Authorization back. 2) for the browser to consider returning a Kerberos Authorization header to the server, additional conditions depend on the browser. For IE : a) the enable Windows Integrated Authentication setting must be on (checked), whether this is done locally by the user, or part of the standard IE settings company-wide, or imposed by some network policy at corporate level. b) the server to which the browser is talking, must be known to IE as either - part of the Intranet - or at least a trusted server That is defined in IE's security zones (which again can be local, or corporation-wide). If condition (a) is not met, when the server sends a 401 Negociate, IE will fall back to NTLM, always. And there is nothing you can do about that at the server level. (Funnily enough, disabling the enable Windows Integrated Authentication at the IE level, has the effect of disabling Kerberos, but not NTLM). If condition (b) is not met, IE will try neither Kerberos nor NTLM, and it /might/ fall back to Basic authentication, if its other settings allow that. That's when you see the browser popup login dialog; and in an SSO context, this is a sure sign that something isn't working as expected. Some authentication modules, at the server level, are able to adapt to what the browser sends, others not. I believe that Waffle can accept either browser NTLM or Kerberos authentication. Waffle works only on a Windows Tomcat server, not on a Linux Tomcat server. I do not know about the SPNEGO thing in Tomcat (from the name, it should). The Jespa module from www.ioplex.com does not handle Kerberos, just NTLM, but it works under both Windows and Linux. And finally, about your problems : it seems that you have fallen in a very specific kind of hell, because you are trying to talk to a Windows-based Kerberos KDC (which is using Windows Kerberos libraries and encryption method choices and hostname formats etc..), from a Java JVM-based client (in this case the Tomcat server, whatever its underlying platform is), which is using Java Kerberos libraries and encryption method choices etc... And it seems that between this Java Kerberos part and the Windows Kerberos part, there are a number of areas of mutual incomprehension (such as which key encryption methods they each implement, or which ones are the default ones for each). And I am sure that the issue can be resolved. But it is probably a question of finding out which among the 25 or more settings one can alter on each side, overlap and either agree or contradict eachother. One underlying issue is that, as well in corporations as on the WWW, the Windows people and the Linux people tend to be 2 separate groups. If you ask the Windows people how to set this up, they will tell you just do this and it works (assuming that all the moving parts are Windows-based); and if you ask the Linux people, they will tell you just do this and it works (assuming that all the moving parts are Linux-based). And there are very few people (and web pages) which span both worlds with their various combinations. David Date: Thu, 26 Mar 2015 09:00:22 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp On 26/03/2015 00:36, David Marsh wrote: Still getting :- java.security.PrivilegedActionException: GSSException: Defective token detected (Mechanism level: G SSHeader did not find the right tag) Folks here mention lack of NegoEx support or bugs in GSS-APi ? http://sourceforge.net/p/spnego/discussion/1003769/thread/990913cc/?page=1 Does Tomcat 8 work with NegoEx ? Is Windows 8.1 and Windows Server 2012 RC2 supported ? My test environment is Windows 2008 R2 server and Windows 7. It is certainly possibly security has been tightened between those versions and 2012/R2 + 8 that means things don't work by default with Java. I'll see if I can find some time in the next few weeks to update my test environment and do some more testing. Mark
Re: SPNEGO test configuration with Manager webapp
On 26/03/2015 00:36, David Marsh wrote: Still getting :- java.security.PrivilegedActionException: GSSException: Defective token detected (Mechanism level: G SSHeader did not find the right tag) Folks here mention lack of NegoEx support or bugs in GSS-APi ? http://sourceforge.net/p/spnego/discussion/1003769/thread/990913cc/?page=1 Does Tomcat 8 work with NegoEx ? Is Windows 8.1 and Windows Server 2012 RC2 supported ? My test environment is Windows 2008 R2 server and Windows 7. It is certainly possibly security has been tightened between those versions and 2012/R2 + 8 that means things don't work by default with Java. I'll see if I can find some time in the next few weeks to update my test environment and do some more testing. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: SPNEGO test configuration with Manager webapp
Hi Mark, Thanks that would be great ! Do you have a good mechanism to test and ensure kerberos token is passed to tomcat and not NTLM token ? David Date: Thu, 26 Mar 2015 09:00:22 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp On 26/03/2015 00:36, David Marsh wrote: Still getting :- java.security.PrivilegedActionException: GSSException: Defective token detected (Mechanism level: G SSHeader did not find the right tag) Folks here mention lack of NegoEx support or bugs in GSS-APi ? http://sourceforge.net/p/spnego/discussion/1003769/thread/990913cc/?page=1 Does Tomcat 8 work with NegoEx ? Is Windows 8.1 and Windows Server 2012 RC2 supported ? My test environment is Windows 2008 R2 server and Windows 7. It is certainly possibly security has been tightened between those versions and 2012/R2 + 8 that means things don't work by default with Java. I'll see if I can find some time in the next few weeks to update my test environment and do some more testing. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: SPNEGO test configuration with Manager webapp
Hi Mark, Thanks for that, yes I've got 30 years windows experience, I can use Linux at a push but its not really my area expertise. I'm a Java / Windows programmer so I should be able to understand it, but not kerberos or Active Directory expert. I have used Waffle in the past with success and used JAAS/GSS-API in Java thick clients. I made the IE settings you outlined but it seems to still prompt. IE has win-tc01.kerbtest.local as a trusted site. Enable Windows Integrated Authentication is on Auto logon only in Intranet Zone is on I've been using Firefox to test and that does send 401 and negotiate, but causes the GSS token error mentioned. Active directory and krb5.ini are using eType 23 which is rc4-hmac The windows client OS and tomcat server OS has registry setting for allowtgtsessionkey set to 1 (enabled). Java kinit test works and stores a ticket in the Java session cache. So problem seems to be either :- 1. Browser sends bad token 2. Token is good but Oracle JDK 8 GSS-API cannot handle it many thanks David Date: Thu, 26 Mar 2015 11:32:39 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks that would be great ! Do you have a good mechanism to test and ensure kerberos token is passed to tomcat and not NTLM token ? I believe that I can answer that. And the basic answer is no. First the basic principle, valid for this and many many other areas : the server cannot impose anything on the browser. The local user can always override anything received from the server, by a setting in the browser. And a hacker can of course do anything. All the server can do, is tell the browser what it will accept, and the browser can tell the server ditto. So, never assume the opposite, and you will save yourself a lot of fruitless searches and dead-ends. Now more specific : 1) For Kerberos to be used at all at the browser level, the server must send a 401 response with Negociate as the requested authentication method. Unless it does that, the browser will never even attempt to send a Kerberos Authorization back. 2) for the browser to consider returning a Kerberos Authorization header to the server, additional conditions depend on the browser. For IE : a) the enable Windows Integrated Authentication setting must be on (checked), whether this is done locally by the user, or part of the standard IE settings company-wide, or imposed by some network policy at corporate level. b) the server to which the browser is talking, must be known to IE as either - part of the Intranet - or at least a trusted server That is defined in IE's security zones (which again can be local, or corporation-wide). If condition (a) is not met, when the server sends a 401 Negociate, IE will fall back to NTLM, always. And there is nothing you can do about that at the server level. (Funnily enough, disabling the enable Windows Integrated Authentication at the IE level, has the effect of disabling Kerberos, but not NTLM). If condition (b) is not met, IE will try neither Kerberos nor NTLM, and it /might/ fall back to Basic authentication, if its other settings allow that. That's when you see the browser popup login dialog; and in an SSO context, this is a sure sign that something isn't working as expected. Some authentication modules, at the server level, are able to adapt to what the browser sends, others not. I believe that Waffle can accept either browser NTLM or Kerberos authentication. Waffle works only on a Windows Tomcat server, not on a Linux Tomcat server. I do not know about the SPNEGO thing in Tomcat (from the name, it should). The Jespa module from www.ioplex.com does not handle Kerberos, just NTLM, but it works under both Windows and Linux. And finally, about your problems : it seems that you have fallen in a very specific kind of hell, because you are trying to talk to a Windows-based Kerberos KDC (which is using Windows Kerberos libraries and encryption method choices and hostname formats etc..), from a Java JVM-based client (in this case the Tomcat server, whatever its underlying platform is), which is using Java Kerberos libraries and encryption method choices etc... And it seems that between this Java Kerberos part and the Windows Kerberos part, there are a number of areas of mutual incomprehension (such as which key encryption methods they each implement, or which ones are the default ones for each). And I am sure that the issue can be resolved. But it is probably a question of finding out which among the 25 or more settings one can alter on each side, overlap and either agree or contradict eachother. One underlying issue is that, as well in corporations as on the WWW, the Windows people and the Linux people tend to be 2 separate groups. If you ask
Re: SPNEGO test configuration with Manager webapp
David Marsh wrote: Hi Mark, Thanks for that, yes I've got 30 years windows experience, I can use Linux at a push but its not really my area expertise. I'm a Java / Windows programmer so I should be able to understand it, but not kerberos or Active Directory expert. I have used Waffle in the past with success and used JAAS/GSS-API in Java thick clients. I made the IE settings you outlined but it seems to still prompt. IE has win-tc01.kerbtest.local as a trusted site. Enable Windows Integrated Authentication is on Auto logon only in Intranet Zone is on I've been using Firefox to test and that does send 401 and negotiate, but causes the GSS token error mentioned. Active directory and krb5.ini are using eType 23 which is rc4-hmac The windows client OS and tomcat server OS has registry setting for allowtgtsessionkey set to 1 (enabled). Java kinit test works and stores a ticket in the Java session cache. So problem seems to be either :- 1. Browser sends bad token 2. Token is good but Oracle JDK 8 GSS-API cannot handle it Another shot almost in the dark : while browsing hundreds of Kerberos-related pages on the WWW, one other recommendation which seems to appear regularly (and Mark also mentioned that somewhere), is that each time you make a change somewhere, you should reboot the machine afterward, before re-testing. (Particularly on Windows machines). I know it's a PITA, but I have also found the same to be true sometimes when merely dealing with NTLM matters. There are probably some hidden caches that get cleared only in that way. many thanks David Date: Thu, 26 Mar 2015 11:32:39 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: SPNEGO test configuration with Manager webapp David Marsh wrote: Hi Mark, Thanks that would be great ! Do you have a good mechanism to test and ensure kerberos token is passed to tomcat and not NTLM token ? I believe that I can answer that. And the basic answer is no. First the basic principle, valid for this and many many other areas : the server cannot impose anything on the browser. The local user can always override anything received from the server, by a setting in the browser. And a hacker can of course do anything. All the server can do, is tell the browser what it will accept, and the browser can tell the server ditto. So, never assume the opposite, and you will save yourself a lot of fruitless searches and dead-ends. Now more specific : 1) For Kerberos to be used at all at the browser level, the server must send a 401 response with Negociate as the requested authentication method. Unless it does that, the browser will never even attempt to send a Kerberos Authorization back. 2) for the browser to consider returning a Kerberos Authorization header to the server, additional conditions depend on the browser. For IE : a) the enable Windows Integrated Authentication setting must be on (checked), whether this is done locally by the user, or part of the standard IE settings company-wide, or imposed by some network policy at corporate level. b) the server to which the browser is talking, must be known to IE as either - part of the Intranet - or at least a trusted server That is defined in IE's security zones (which again can be local, or corporation-wide). If condition (a) is not met, when the server sends a 401 Negociate, IE will fall back to NTLM, always. And there is nothing you can do about that at the server level. (Funnily enough, disabling the enable Windows Integrated Authentication at the IE level, has the effect of disabling Kerberos, but not NTLM). If condition (b) is not met, IE will try neither Kerberos nor NTLM, and it /might/ fall back to Basic authentication, if its other settings allow that. That's when you see the browser popup login dialog; and in an SSO context, this is a sure sign that something isn't working as expected. Some authentication modules, at the server level, are able to adapt to what the browser sends, others not. I believe that Waffle can accept either browser NTLM or Kerberos authentication. Waffle works only on a Windows Tomcat server, not on a Linux Tomcat server. I do not know about the SPNEGO thing in Tomcat (from the name, it should). The Jespa module from www.ioplex.com does not handle Kerberos, just NTLM, but it works under both Windows and Linux. And finally, about your problems : it seems that you have fallen in a very specific kind of hell, because you are trying to talk to a Windows-based Kerberos KDC (which is using Windows Kerberos libraries and encryption method choices and hostname formats etc..), from a Java JVM-based client (in this case the Tomcat server, whatever its underlying platform is), which is using Java Kerberos libraries and encryption method choices etc... And it seems that between this Java Kerberos part and the Windows Kerberos part, there are a number of areas of mutual incomprehension (such as which key encryption methods
Re: SPNEGO test configuration with Manager webapp
David Marsh wrote:
Put keytab in c:\keytab\tomcat.keytab, ensured owner was tc01@KERTEST.LOCAL,
still same symptoms.
Ran klist on client after firefox test and the three 401 responses. :-
C:\Users\test.KERBTEST.000klist
Current LogonId is 0:0x2fd7a
Cached Tickets: (2)
#0 Client: test @ KERBTEST.LOCAL
Server: krbtgt/KERBTEST.LOCAL @ KERBTEST.LOCAL
KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
Ticket Flags 0x40e1 - forwardable renewable initial pre_authent nam
e_canonicalize
Start Time: 3/25/2015 14:46:43 (local)
End Time: 3/26/2015 0:46:43 (local)
Renew Time: 4/1/2015 14:46:43 (local)
Session Key Type: AES-256-CTS-HMAC-SHA1-96
Cache Flags: 0x1 - PRIMARY
Kdc Called: 192.168.0.200
#1 Client: test @ KERBTEST.LOCAL
Server: HTTP/win-tc01.kerbtest.local @ KERBTEST.LOCAL
KerbTicket Encryption Type: RSADSI RC4-HMAC(NT)
Ticket Flags 0x40a1 - forwardable renewable pre_authent name_canoni
calize
Start Time: 3/25/2015 14:51:21 (local)
End Time: 3/26/2015 0:46:43 (local)
Renew Time: 4/1/2015 14:46:43 (local)
Session Key Type: RSADSI RC4-HMAC(NT)
Cache Flags: 0
Kdc Called: 192.168.0.200
Looks like I was granted a ticket for the SPN HTTP/win-tc01.kerbtest.local @
KERBTEST.LOCAL ?
If I have ticket why do I get 401 ?
Maybe because these things come from 2 different places ?
- ticket # 0 is a general ticket-granting ticket (krbtgt) obtained by the client
directly from the KDC
- ticket # 1 is a ticket to access HTTP/Tomcat, obtained by the client directly from the
KDC (after presenting his ticket-granting ticket)
- the 401 response is a response from Tomcat, when the client tries to access it by
presenting his HTTP/Tomcat ticket
So the problem could be that Tomcat is unable to validate the client ticket, for some
reason proper to Tomcat itself, not to the client ticket per se (which is probably valid)
Again, in your (presumably Tomcat) Kerberos log, it looked as if Tomcat was having trouble
pre-authenticating itself, whatever that means. Maybe such a succesful
pre-authentication is a pre-requisite for Tomcat to be able to recognise client tickets to
itself ?
Date: Tue, 24 Mar 2015 22:46:15 +
From: ma...@apache.org
To: users@tomcat.apache.org
Subject: Re: SPNEGO test configuration with Manager webapp
On 24/03/2015 20:47, David Marsh wrote:
Hi Felix,
Thanks fort your help!
I have enabled krb5 and gss debug.I altered CATALINA_OPTS in startup.bat and
also added the same definitions to the Java parameters in Configure Tomcat
tool.I definitely got more information when using startup.bat, not sure the
settings get picked up by the windows service ?
I do not think authentication completes, certainly authorization does not as I
cant see the site and get 401 http status.
I have not configured a tomcat realm but I have put the test user a manager-gui
group in Active Directory.
I've only given your config a quick scan, but the thing that jumps out
at me is spaces in the some of the paths. I'm not sure how well krb5.ini
will handle those. It might be fine. It might not be.
Mark
David
Date: Tue, 24 Mar 2015 21:39:38 +0100
From: felix.schumac...@internetallee.de
To: users@tomcat.apache.org
Subject: Re: SPNEGO test configuration with Manager webapp
Am 24.03.2015 um 21:25 schrieb David Marsh:
Everything is as described and still not working, except the jaas.conf is :-
com.sun.security.jgss.krb5.initiate {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
principal=HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL
useKeyTab=true
keyTab=C:/Program Files/Apache Software Foundation/Tomcat
8.0/conf/tomcat.keytab
storeKey=true;
};
com.sun.security.jgss.krb5.accept {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
principal=HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL
useKeyTab=true
keyTab=C:/Program Files/Apache Software Foundation/Tomcat
8.0/conf/tomcat.keytab
storeKey=true;
};
In other words the principal is the tomcat server as it should be.
Date: Tue, 24 Mar 2015 21:17:59 +0100
From: felix.schumac...@internetallee.de
To: users@tomcat.apache.org
Subject: Re: SPNEGO test configuration with Manager webapp
Am 24.03.2015 um 21:05 schrieb David Marsh:
Sorry thats :-
principal=HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL
under jaas.conf, it is set to the tomcat server DNS.
Is it working with this configuration, or just to point out, that you
copied the wrong jaas.conf for the mail?
Felix
From: dmars...@outlook.com
To: users@tomcat.apache.org
Subject: SPNEGO test configuration with Manager webapp
Date: Tue, 24 Mar 2015 20:02:04 +
I'm trying to get SPNEGO authentication working with Tomcat 8.
I've created three Windows VMs :-
Tomcat Server - Windows 8.1 32 bit VM
Test Client - Windows 8.1 32
RE: SPNEGO test configuration with Manager webapp
Put keytab in c:\keytab\tomcat.keytab, ensured owner was tc01@KERTEST.LOCAL,
still same symptoms.
Ran klist on client after firefox test and the three 401 responses. :-
C:\Users\test.KERBTEST.000klist
Current LogonId is 0:0x2fd7a
Cached Tickets: (2)
#0 Client: test @ KERBTEST.LOCAL
Server: krbtgt/KERBTEST.LOCAL @ KERBTEST.LOCAL
KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
Ticket Flags 0x40e1 - forwardable renewable initial pre_authent nam
e_canonicalize
Start Time: 3/25/2015 14:46:43 (local)
End Time: 3/26/2015 0:46:43 (local)
Renew Time: 4/1/2015 14:46:43 (local)
Session Key Type: AES-256-CTS-HMAC-SHA1-96
Cache Flags: 0x1 - PRIMARY
Kdc Called: 192.168.0.200
#1 Client: test @ KERBTEST.LOCAL
Server: HTTP/win-tc01.kerbtest.local @ KERBTEST.LOCAL
KerbTicket Encryption Type: RSADSI RC4-HMAC(NT)
Ticket Flags 0x40a1 - forwardable renewable pre_authent name_canoni
calize
Start Time: 3/25/2015 14:51:21 (local)
End Time: 3/26/2015 0:46:43 (local)
Renew Time: 4/1/2015 14:46:43 (local)
Session Key Type: RSADSI RC4-HMAC(NT)
Cache Flags: 0
Kdc Called: 192.168.0.200
Looks like I was granted a ticket for the SPN HTTP/win-tc01.kerbtest.local @
KERBTEST.LOCAL ?
If I have ticket why do I get 401 ?
Date: Tue, 24 Mar 2015 22:46:15 +
From: ma...@apache.org
To: users@tomcat.apache.org
Subject: Re: SPNEGO test configuration with Manager webapp
On 24/03/2015 20:47, David Marsh wrote:
Hi Felix,
Thanks fort your help!
I have enabled krb5 and gss debug.I altered CATALINA_OPTS in startup.bat and
also added the same definitions to the Java parameters in Configure Tomcat
tool.I definitely got more information when using startup.bat, not sure the
settings get picked up by the windows service ?
I do not think authentication completes, certainly authorization does not as
I cant see the site and get 401 http status.
I have not configured a tomcat realm but I have put the test user a
manager-gui group in Active Directory.
I've only given your config a quick scan, but the thing that jumps out
at me is spaces in the some of the paths. I'm not sure how well krb5.ini
will handle those. It might be fine. It might not be.
Mark
David
Date: Tue, 24 Mar 2015 21:39:38 +0100
From: felix.schumac...@internetallee.de
To: users@tomcat.apache.org
Subject: Re: SPNEGO test configuration with Manager webapp
Am 24.03.2015 um 21:25 schrieb David Marsh:
Everything is as described and still not working, except the jaas.conf is
:-
com.sun.security.jgss.krb5.initiate {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
principal=HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL
useKeyTab=true
keyTab=C:/Program Files/Apache Software Foundation/Tomcat
8.0/conf/tomcat.keytab
storeKey=true;
};
com.sun.security.jgss.krb5.accept {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
principal=HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL
useKeyTab=true
keyTab=C:/Program Files/Apache Software Foundation/Tomcat
8.0/conf/tomcat.keytab
storeKey=true;
};
In other words the principal is the tomcat server as it should be.
Date: Tue, 24 Mar 2015 21:17:59 +0100
From: felix.schumac...@internetallee.de
To: users@tomcat.apache.org
Subject: Re: SPNEGO test configuration with Manager webapp
Am 24.03.2015 um 21:05 schrieb David Marsh:
Sorry thats :-
principal=HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL
under jaas.conf, it is set to the tomcat server DNS.
Is it working with this configuration, or just to point out, that you
copied the wrong jaas.conf for the mail?
Felix
From: dmars...@outlook.com
To: users@tomcat.apache.org
Subject: SPNEGO test configuration with Manager webapp
Date: Tue, 24 Mar 2015 20:02:04 +
I'm trying to get SPNEGO authentication working with Tomcat 8.
I've created three Windows VMs :-
Tomcat Server - Windows 8.1 32 bit VM
Test Client - Windows 8.1 32 bit VM
Domain Controller - Windows Server 2012 R2 64 bit VM
The Tomcat Server and the Test Client are joined to the same domain
kerbtest.local, they are logged in with domain logins.
The firewall is disabled on the Tomcat Server VM.
I've followed the guidelines on the Apache Tomcat website.
jaas.conf
com.sun.security.jgss.krb5.initiate {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
principal=HTTP/win-dc01.kerbtest.local@KERBTEST.LOCAL
useKeyTab=true
keyTab=C:/Program Files/Apache Software Foundation/Tomcat
8.0/conf/tomcat.keytab
storeKey=true;
};
com.sun.security.jgss.krb5.accept {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
principal=HTTP/win-dc01.kerbtest.local@KERBTEST.LOCAL
useKeyTab=true
keyTab=C:/Program Files/Apache Software Foundation
RE: SPNEGO test configuration with Manager webapp
Am 25.03.2015 16:09, schrieb David Marsh:
Put keytab in c:\keytab\tomcat.keytab, ensured owner was
tc01@KERTEST.LOCAL, still same symptoms.
Ran klist on client after firefox test and the three 401 responses. :-
C:\Users\test.KERBTEST.000klist
Current LogonId is 0:0x2fd7a
Cached Tickets: (2)
#0 Client: test @ KERBTEST.LOCAL
Server: krbtgt/KERBTEST.LOCAL @ KERBTEST.LOCAL
KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
Ticket Flags 0x40e1 - forwardable renewable initial
pre_authent nam
e_canonicalize
Start Time: 3/25/2015 14:46:43 (local)
End Time: 3/26/2015 0:46:43 (local)
Renew Time: 4/1/2015 14:46:43 (local)
Session Key Type: AES-256-CTS-HMAC-SHA1-96
Cache Flags: 0x1 - PRIMARY
Kdc Called: 192.168.0.200
#1 Client: test @ KERBTEST.LOCAL
Server: HTTP/win-tc01.kerbtest.local @ KERBTEST.LOCAL
KerbTicket Encryption Type: RSADSI RC4-HMAC(NT)
Ticket Flags 0x40a1 - forwardable renewable pre_authent
name_canoni
calize
Start Time: 3/25/2015 14:51:21 (local)
End Time: 3/26/2015 0:46:43 (local)
Renew Time: 4/1/2015 14:46:43 (local)
Session Key Type: RSADSI RC4-HMAC(NT)
Cache Flags: 0
Kdc Called: 192.168.0.200
Looks like I was granted a ticket for the SPN
HTTP/win-tc01.kerbtest.local @ KERBTEST.LOCAL ?
If I have ticket why do I get 401 ?
Your client has got a service ticket for HTTP/win-tc01... This is used
by firefox for authentication. Firefox transmits
this service ticket to the server (as base64 encoded in the
WWW-Authenticate header).
Your server has to decrypt this ticket using its own ticket to get at
the user information. This is where your problems arise.
It looks like your server has trouble to get its own ticket.
Are you sure, that the password you used for keytab generation (on the
server side), is correct? ktpass will probably accept
any input as a password. Maybe you can check the keytab by using kinit
(though I don't know, if it exists for windows, or how
the java one is used).
Felix
Date: Tue, 24 Mar 2015 22:46:15 +
From: ma...@apache.org
To: users@tomcat.apache.org
Subject: Re: SPNEGO test configuration with Manager webapp
On 24/03/2015 20:47, David Marsh wrote:
Hi Felix,
Thanks fort your help!
I have enabled krb5 and gss debug.I altered CATALINA_OPTS in
startup.bat and also added the same definitions to the Java
parameters in Configure Tomcat tool.I definitely got more information
when using startup.bat, not sure the settings get picked up by the
windows service ?
I do not think authentication completes, certainly authorization does
not as I cant see the site and get 401 http status.
I have not configured a tomcat realm but I have put the test user a
manager-gui group in Active Directory.
I've only given your config a quick scan, but the thing that jumps out
at me is spaces in the some of the paths. I'm not sure how well
krb5.ini
will handle those. It might be fine. It might not be.
Mark
David
Date: Tue, 24 Mar 2015 21:39:38 +0100
From: felix.schumac...@internetallee.de
To: users@tomcat.apache.org
Subject: Re: SPNEGO test configuration with Manager webapp
Am 24.03.2015 um 21:25 schrieb David Marsh:
Everything is as described and still not working, except the
jaas.conf is :-
com.sun.security.jgss.krb5.initiate {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
principal=HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL
useKeyTab=true
keyTab=C:/Program Files/Apache Software Foundation/Tomcat
8.0/conf/tomcat.keytab
storeKey=true;
};
com.sun.security.jgss.krb5.accept {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
principal=HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL
useKeyTab=true
keyTab=C:/Program Files/Apache Software Foundation/Tomcat
8.0/conf/tomcat.keytab
storeKey=true;
};
In other words the principal is the tomcat server as it should be.
Date: Tue, 24 Mar 2015 21:17:59 +0100
From: felix.schumac...@internetallee.de
To: users@tomcat.apache.org
Subject: Re: SPNEGO test configuration with Manager webapp
Am 24.03.2015 um 21:05 schrieb David Marsh:
Sorry thats :-
principal=HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL
under jaas.conf, it is set to the tomcat server DNS.
Is it working with this configuration, or just to point out, that
you
copied the wrong jaas.conf for the mail?
Felix
From: dmars...@outlook.com
To: users@tomcat.apache.org
Subject: SPNEGO test configuration with Manager webapp
Date: Tue, 24 Mar 2015 20:02:04 +
I'm trying to get SPNEGO authentication working with Tomcat 8.
I've created three Windows VMs :-
Tomcat Server - Windows 8.1 32 bit VM
Test Client - Windows 8.1 32 bit VM
Domain Controller - Windows Server 2012 R2 64 bit VM
The Tomcat Server and the Test Client are joined to the same
domain kerbtest.local
RE: SPNEGO test configuration with Manager webapp
Its possible I guess, although I would not expect that. The test is :- Client Test Windows 8.1 VM with Firefox - Tomcat Server Windows 8.1 VM Firefox is not configured to use a proxy, its all in Vmware Workstation 10 using the Vmnet01 virtual network. Firefox has three 401 responses with headers Authorization and WWW-Authenticate :- 1 :- Reponse WWW-Authenticate: Negotiate 2 :- Request Authorization: Negotiate YIIGUgYGKwYBBQUCoIIGRjCCBkKgMDAuBgkqhkiC9xIBAgIGCSqGSIb3EgECAgYKKwYBBAGCNwICHgYKKwYBBAGCNwICCqKCBgwEggYIYIIGBAYJKoZIhvcSAQICAQBuggXzMIIF76ADAgEFoQMCAQ6iBwMFACCjggR6YYIEdjCCBHKgAwIBBaEQGw5LRVJCVEVTVC5MT0NBTKIqMCigAwIBAqEhMB8bBEhUVFAbF3dpbi10YzAxLmtlcmJ0ZXN0LmxvY2Fso4IEKzCCBCegAwIBF6EDAgEDooIEGQSCBBVToJwn2tPBboTTk5BBzJktj/GIuSekyM94atYd2nmQZr+LRVHUS1CD27iufu9aGtRLNT2YStbH3VgBpxcB0mEdOGcqfwif2htDkbFbSr6bmvZLz7PDMZv0mpUw2jcLnuVYpJjcw0fygonPpLYNTKnwrJJQA7eYMqY5DWI2ntF5RACw0qHJrXY2yFBQ3GOo8+1PHz9WcuxmTdUsLgx9QbFvEjTdksor5xvsInRNWOdjwgObnnhzGEF2RbAyD3HYanU4pdK9QL7HIEL5AI61czl2RfgVzDIGokBlW3k6R7jEp6jUBOwBjTnJC8gZthlAfTIqRlyZOntbFeHboeNY6YYtFukdewgBSuFKRTPd7wv4cvSBrF+FsvwIM0wiy2Kkp6fvyh3O/fHRXSR5AaJvnbIj+XtIUX86K5TGG0GmA9hnLjt4sacfxxz05aqlpQ1ttPBt67MEMECQiZZB4Ck1BsMpLSf22tCSVUwZEZF0MdtKiQTe7U0GDOEcm5oZfhpn8ecDkEosinyk10jGFK1cyr23TcwIlLH6yC0YaksB19EAADSF9dQKbftRUVcTjUgOdGcf7eEcUdNcmYw/ftHsanMwZEat5lznurgVFDwa6rjxVoc+X/C6Dwl+ME/yEClpwn6bxxDyCssxUgYsiRfWJGCr6EEPdWB5omQUf1o9ArvEbgtyS4kkHGLa3X5FeXctRwi2Yj/uLYnEOZHfkcoKk31FvdhSr92Kry4926hlS9ao4nyGS7ZVnvr1n8r5V6+D6UbYhUQgBvEaERgc8T822kiij1N/szQePAze4YWWTA0djryRSB0qqMGgBdtzg76+whlvjOkG0J4MjUbFy1iLvfOkIWXgHRChGeMCrphv64NmfgHQmOiYPdqtTgYlAvyW9riL1kci7Xz+D1XwfxJpdimsakfyRqpjIEkgU+QEN+aL8/1X8lRTu8uTepXVReBlSx2Am+DFgesBlkjWuYmIuj84mUH0Lcc7yHytOyfO5OJ4mI5O5YNkl167xMcI9akaH7LtS+c1OnfHwtlJsatLnOyLYwYP9KWpkh0i2d4DNV0EYs3B68UbsY3f4+bZcHW9SQ/PthGjzk5FTdOKh5dD0BLf1ADl+Rp5hegl0iGS6cVpZFnu8n3wPd2eenwQn0EDvyx3nuMyeETqqXEuLjTbqbMpzIxSxFl5s/1Nwaf4Up0a8wcEDNj3acnHicis8ELEORo+wtJnd0wyMIpfC+tFRsewhEHDttjWnqxkHbfpbOnChZkLOL04YoflhHK3ZrsBXk0Yu0udKIZBoJ7Pf5qiOdE36lEjAkWLB/2wVD+zvxfIKd7r9FSxAfYz0UsVYVyBX0RtF5GCpTPqLAk9ImL4xxpkijpUUwjlM9WylH8jafaHGwfmpUM9pIIBWjCCAVagAwIBF6KCAU0EggFJv04NvH3OA0+sXGdCWanthHZBM9DIq0AknWszbwm9z+7da/DThLEAnnozvO84tK/DD7fC/AnSWKXnqchILMdjPnZA5Bg3yjS4Y1rJFawc9fDNUmTCn4ILjjl6SSETMbJSFjzarv4wEfy5VU16DNBzWUxEJNH8PvsXTTfdzcwdsYnFwHGZbrcNxaJUtp3xpyoG/1EAgNk9i1UtewL1bHVkmmuJXUXXetL7v4RzMuVD5q68q8nWDB1toKgcEjHEgEHWjODwSD/zoYwZrn1nCtnRm8aN9xKr097iK5K8ZUJKxWr4SlmAI6tZSyaVJGWJSzRvb47SZ9TVfk6Xft+vV+pVjxXdNAKIqHqA4tUfPCKgWff6iGmQI4fnJG5yYyyNFXOajz0qMYpfnbNLjc+nhsxjOUvZKOT4xTvhuOTCmdtabMybTVx4uNJEQ/4= Response WWW-Authenticate: Negotiate oRQwEqADCgEBoQsGCSqGSIb3EgECAg== 3 :- Request Authorization: Negotiate oYIGGTCCBhWgAwoBAaKCBgwEggYIYIIGBAYJKoZIhvcSAQICAQBuggXzMIIF76ADAgEFoQMCAQ6iBwMFACCjggR6YYIEdjCCBHKgAwIBBaEQGw5LRVJCVEVTVC5MT0NBTKIqMCigAwIBAqEhMB8bBEhUVFAbF3dpbi10YzAxLmtlcmJ0ZXN0LmxvY2Fso4IEKzCCBCegAwIBF6EDAgEDooIEGQSCBBVToJwn2tPBboTTk5BBzJktj/GIuSekyM94atYd2nmQZr+LRVHUS1CD27iufu9aGtRLNT2YStbH3VgBpxcB0mEdOGcqfwif2htDkbFbSr6bmvZLz7PDMZv0mpUw2jcLnuVYpJjcw0fygonPpLYNTKnwrJJQA7eYMqY5DWI2ntF5RACw0qHJrXY2yFBQ3GOo8+1PHz9WcuxmTdUsLgx9QbFvEjTdksor5xvsInRNWOdjwgObnnhzGEF2RbAyD3HYanU4pdK9QL7HIEL5AI61czl2RfgVzDIGokBlW3k6R7jEp6jUBOwBjTnJC8gZthlAfTIqRlyZOntbFeHboeNY6YYtFukdewgBSuFKRTPd7wv4cvSBrF+FsvwIM0wiy2Kkp6fvyh3O/fHRXSR5AaJvnbIj+XtIUX86K5TGG0GmA9hnLjt4sacfxxz05aqlpQ1ttPBt67MEMECQiZZB4Ck1BsMpLSf22tCSVUwZEZF0MdtKiQTe7U0GDOEcm5oZfhpn8ecDkEosinyk10jGFK1cyr23TcwIlLH6yC0YaksB19EAADSF9dQKbftRUVcTjUgOdGcf7eEcUdNcmYw/ftHsanMwZEat5lznurgVFDwa6rjxVoc+X/C6Dwl+ME/yEClpwn6bxxDyCssxUgYsiRfWJGCr6EEPdWB5omQUf1o9ArvEbgtyS4kkHGLa3X5FeXctRwi2Yj/uLYnEOZHfkcoKk31FvdhSr92Kry4926hlS9ao4nyGS7ZVnvr1n8r5V6+D6UbYhUQgBvEaERgc8T822kiij1N/szQePAze4YWWTA0djryRSB0qqMGgBdtzg76+whlvjOkG0J4MjUbFy1iLvfOkIWXgHRChGeMCrphv64NmfgHQmOiYPdqtTgYlAvyW9riL1kci7Xz+D1XwfxJpdimsakfyRqpjIEkgU+QEN+aL8/1X8lRTu8uTepXVReBlSx2Am+DFgesBlkjWuYmIuj84mUH0Lcc7yHytOyfO5OJ4mI5O5YNkl167xMcI9akaH7LtS+c1OnfHwtlJsatLnOyLYwYP9KWpkh0i2d4DNV0EYs3B68UbsY3f4+bZcHW9SQ/PthGjzk5FTdOKh5dD0BLf1ADl+Rp5hegl0iGS6cVpZFnu8n3wPd2eenwQn0EDvyx3nuMyeETqqXEuLjTbqbMpzIxSxFl5s/1Nwaf4Up0a8wcEDNj3acnHicis8ELEORo+wtJnd0wyMIpfC+tFRsewhEHDttjWnqxkHbfpbOnChZkLOL04YoflhHK3ZrsBXk0Yu0udKIZBoJ7Pf5qiOdE36lEjAkWLB/2wVD+zvxfIKd7r9FSxAfYz0UsVYVyBX0RtF5GCpTPqLAk9ImL4xxpkijpUUwjlM9WylH8jafaHGwfmpUM9pIIBWjCCAVagAwIBF6KCAU0EggFJxK5PpTX/g5phbQ2bv8XrnUCfC+cfDkPjAOnpnsiX7fRtA7k5qaEtUI/9KlqcAbV0jG3nQolKK5zEL6ftBXPW3FgZRRGmiYMQVpjBtIKapE1A+V/dveIrnnkxuuRmWrIJFYagOijzyilZj6cIIJqtmqI+QE4vKGIQl6lMwcgao9ZNZ2t2vLI5cD/BSjkFNbmgqLAuDZW357KVd5uoUJbHDpQHGWKw4A4x9vpvv+NUv1IrUaBe19PDQup/SILLHlUA8zr/OsHMytfPpVSv99fLBY7mcr0zwm+qhPF9Pos+Ch8y4hkocVOMXKEOcF+AKbxrzYhOydMFqanW6vNYQqB7Azz3GtP0YkFhU38JBG9UeKinEw2KT1Ii2pjCmTlF3/Q7gG2uqw6T5DR452ffxipG4yvXMCebDCnetitAbeIPXFJv1hdaJuMCO2E= Reponse WWW-Authenticate: Negotiate I'm not sure how long they should be, but they all end = so expect not truncated ? Subject: RE: SPNEGO test configuration
RE: SPNEGO test configuration with Manager webapp
ator.authenticate No authorization header sent by client 25-Mar-2015 15:46:28.305 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Failed authenticate() test 25-Mar-2015 15:46:28.417 FINE [http-nio-80-exec-2] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Security checking request GET /manager/html 25-Mar-2015 15:46:28.420 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Status interface]' against GET /html -- false 25-Mar-2015 15:46:28.422 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[JMX Proxy interface]' against GET /html -- fal se 25-Mar-2015 15:46:28.424 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Text Manager interface (for scripts)]' against GET /html -- false 25-Mar-2015 15:46:28.425 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[HTML Manager interface (for humans)]' against G ET /html -- true 25-Mar-2015 15:46:28.427 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Status interface]' against GET /html -- false 25-Mar-2015 15:46:28.428 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[JMX Proxy interface]' against GET /html -- fal se 25-Mar-2015 15:46:28.429 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Text Manager interface (for scripts)]' against GET /html -- false 25-Mar-2015 15:46:28.442 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[HTML Manager interface (for humans)]' against G ET /html -- true 25-Mar-2015 15:46:28.444 FINE [http-nio-80-exec-2] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Calling hasUserDataPermission() 25-Mar-2015 15:46:28.445 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.hasUserDataPe rmission User data constraint has no restrictions 25-Mar-2015 15:46:28.445 FINE [http-nio-80-exec-2] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Calling authenticate() Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt true ticketCache is nul l isInitiator true KeyTab is C:/keytab/tomcat.keytab refreshKrb5Config is false principal is HTTP/wi n-tc01.kerbtest.local@KERBTEST.LOCAL tryFirstPass is false useFirstPass is false storePass is false clearPass is false KeyTabInputStream, readName(): kerbtest.local KeyTabInputStream, readName(): HTTP KeyTabInputStream, readName(): win-tc01.kerbtest.local KeyTab: load() entry length: 78; type: 23 Looking for keys for: HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL Java config name: C:\Program Files\Apache Software Foundation\Tomcat 8.0\conf\krb5.ini Loaded from Java config Added key: 23version: 3 KdcAccessibility: reset Looking for keys for: HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL Added key: 23version: 3 default etypes for default_tkt_enctypes: 23 18 17. KrbAsReq creating message KrbKdcReq send: kdc=win-dc01.kerbtest.local UDP:88, timeout=3, number of retries =3, #bytes= 164 KDCCommunication: kdc=win-dc01.kerbtest.local UDP:88, timeout=3,Attempt =1, #bytes=164 KrbKdcReq send: #bytes read=185 Pre-Authentication Data: PA-DATA type = 11 PA-ETYPE-INFO etype = 23, salt = Pre-Authentication Data: PA-DATA type = 19 PA-ETYPE-INFO2 etype = 23, salt = null, s2kparams = null Pre-Authentication Data: PA-DATA type = 2 PA-ENC-TIMESTAMP Pre-Authentication Data: PA-DATA type = 16 Pre-Authentication Data: PA-DATA type = 15 KdcAccessibility: remove win-dc01.kerbtest.local:88 KDCRep: init() encoding tag is 126 req type is 11 KRBError: sTime is Wed Mar 25 15:46:28 GMT 2015 1427298388000 suSec is 701709 error code is 25 error Message is Additional pre-authentication required sname is krbtgt/KERBTEST.LOCAL@KERBTEST.LOCAL eData provided. msgType is 30 Pre-Authentication Data: PA-DATA type = 11 PA-ETYPE-INFO etype = 23, salt = Pre-Authentication Data: PA-DATA type = 19 PA-ETYPE-INFO2 etype = 23, salt = null, s2kparams = null Pre-Authentication Data: PA-DATA type = 2 PA-ENC-TIMESTAMP Pre-Authentication Data: PA-DATA type = 16 Pre-Authentication Data: PA-DATA type = 15 KrbAsReqBuilder: PREAUTH FAILED/REQ, re-send AS-REQ default etypes for default_tkt_enctypes: 23 18 17. Looking for keys for: HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL Added key: 23version: 3 Looking for keys for: HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL Added key: 23version: 3 default etypes for default_tkt_enctypes: 23 18 17. EType: sun.security.krb5.internal.crypto.ArcFourHmacEType KrbAsReq creating message KrbKdcReq
RE: SPNEGO test configuration with Manager webapp
se.invoke Calling authenticate() 25-Mar-2015 15:46:28.304 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.SpnegoAuthentic ator.authenticate No authorization header sent by client 25-Mar-2015 15:46:28.305 FINE [http-nio-80-exec-1] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Failed authenticate() test 25-Mar-2015 15:46:28.417 FINE [http-nio-80-exec-2] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Security checking request GET /manager/html 25-Mar-2015 15:46:28.420 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Status interface]' against GET /html -- false 25-Mar-2015 15:46:28.422 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[JMX Proxy interface]' against GET /html -- fal se 25-Mar-2015 15:46:28.424 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Text Manager interface (for scripts)]' against GET /html -- false 25-Mar-2015 15:46:28.425 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[HTML Manager interface (for humans)]' against G ET /html -- true 25-Mar-2015 15:46:28.427 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Status interface]' against GET /html -- false 25-Mar-2015 15:46:28.428 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[JMX Proxy interface]' against GET /html -- fal se 25-Mar-2015 15:46:28.429 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[Text Manager interface (for scripts)]' against GET /html -- false 25-Mar-2015 15:46:28.442 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.findSecurityC onstraints Checking constraint 'SecurityConstraint[HTML Manager interface (for humans)]' against G ET /html -- true 25-Mar-2015 15:46:28.444 FINE [http-nio-80-exec-2] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Calling hasUserDataPermission() 25-Mar-2015 15:46:28.445 FINE [http-nio-80-exec-2] org.apache.catalina.realm.RealmBase.hasUserDataPe rmission User data constraint has no restrictions 25-Mar-2015 15:46:28.445 FINE [http-nio-80-exec-2] org.apache.catalina.authenticator.AuthenticatorBa se.invoke Calling authenticate() Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt true ticketCache is nul l isInitiator true KeyTab is C:/keytab/tomcat.keytab refreshKrb5Config is false principal is HTTP/wi n-tc01.kerbtest.local@KERBTEST.LOCAL tryFirstPass is false useFirstPass is false storePass is false clearPass is false KeyTabInputStream, readName(): kerbtest.local KeyTabInputStream, readName(): HTTP KeyTabInputStream, readName(): win-tc01.kerbtest.local KeyTab: load() entry length: 78; type: 23 Looking for keys for: HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL Java config name: C:\Program Files\Apache Software Foundation\Tomcat 8.0\conf\krb5.ini Loaded from Java config Added key: 23version: 3 KdcAccessibility: reset Looking for keys for: HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL Added key: 23version: 3 default etypes for default_tkt_enctypes: 23 18 17. KrbAsReq creating message KrbKdcReq send: kdc=win-dc01.kerbtest.local UDP:88, timeout=3, number of retries =3, #bytes= 164 KDCCommunication: kdc=win-dc01.kerbtest.local UDP:88, timeout=3,Attempt =1, #bytes=164 KrbKdcReq send: #bytes read=185 Pre-Authentication Data: PA-DATA type = 11 PA-ETYPE-INFO etype = 23, salt = Pre-Authentication Data: PA-DATA type = 19 PA-ETYPE-INFO2 etype = 23, salt = null, s2kparams = null Pre-Authentication Data: PA-DATA type = 2 PA-ENC-TIMESTAMP Pre-Authentication Data: PA-DATA type = 16 Pre-Authentication Data: PA-DATA type = 15 KdcAccessibility: remove win-dc01.kerbtest.local:88 KDCRep: init() encoding tag is 126 req type is 11 KRBError: sTime is Wed Mar 25 15:46:28 GMT 2015 1427298388000 suSec is 701709 error code is 25 error Message is Additional pre-authentication required sname is krbtgt/KERBTEST.LOCAL@KERBTEST.LOCAL eData provided. msgType is 30 Pre-Authentication Data: PA-DATA type = 11 PA-ETYPE-INFO etype = 23, salt = Pre-Authentication Data: PA-DATA type = 19 PA-ETYPE-INFO2 etype = 23, salt = null, s2kparams = null Pre-Authentication Data: PA-DATA type = 2 PA-ENC-TIMESTAMP Pre-Authentication Data: PA-DATA type = 16 Pre-Authentication Data: PA-DATA type = 15 KrbAsReqBuilder: PREAUTH FAILED/REQ, re-send AS-REQ default etypes for default_tkt_enctypes: 23 18 17. Looking for keys for: HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL Added key: 23version: 3 Looking for keys for: HTTP/win-tc01.kerbtest.local@KERBTEST.LOCAL Added key: 23version: 3 default etypes
