Re: slow or timeout with client certificate and some http client against tomcat 8.5 with Nio2 OpenSSL implementation

[Hugh H]

Hi Mark,

Here are the logs you requested

client:
https://1drv.ms/t/s!Aii8T4l0bnqVlyAuRIjSuluBe8vy

server:
https://1drv.ms/u/s!Aii8T4l0bnqVlx-TGo6I0dMXZxG1


I checked the system clock right before my testing and the server and the 
client are synchronized.

Thanks,
Hugh

On May 1, 2018, at 9:31 AM, Mark Thomas 
> wrote:

On 01/05/18 03:11, 旭东 胡 wrote:
Hi Mark,

Unfortunately,  8.5.31 does not resolve my issue. You can find the catalina.out 
log by https://1drv.ms/u/s!Aii8T4l0bnqVlx0mqtHngJ_1OvRo.
From my client log the timeout occurs:
1. between 15:03:48 and 15:04:48
2. between 15:04:48 and 15:05:48
3. between 15:05:49 and 15:06:49
4. between 15:06:59 and 15:07:49
5. between 15:07:59 and 15:08:49
6. between 15:08:59 and 15:09:49

The problematic port is 11443. Sorry there is a health checking, which I cannot 
turned off, on port 10443 adding a lot noise.

OK.

First of all, please ensure that the time on the client and server are
synchronized. Give that the server log doesn't show the server starting
until 15:04:00 the client and server look to be ~25 seconds out of sync.

What I see in most of the connections is the TLS handshake completing
and the I/O layer passing the socket to the protocol layer for
processing. The socket is returned from the protocol layer with an
instruction to close the socket.

We need to see what is happening in the protocol layer. Please add the
following to logging.properties, restart Tomcat 8.5.31 and repeat your test:
org.apache.coyote.level=FINE

Please also include the client logs this time.

Thanks,

Mark



Thanks,
Hugh

On Apr 30, 2018, at 5:08 AM, Mark Thomas 
> wrote:

On 30/04/18 01:48, ** * wrote:
Hi,

I met a weird issue during setting up tomcat 8.5 with Http11Nio2Protocol 
connector and OpenSSLImplementation. The issue is that a request would be 
timeout using apache HttpClient and client certificate after serval previous 
requests. It also happens with RestAssured and SoapUI. Please note it works 
fine for first several requests and then failed with timeout.

However, this issue is not observed when JMeter, tried both JAVA and non-JAVA 
implementation, and insomnia REST client being used. I used a static page to 
rule out application factors. Also Http11NioProtocol works fine for all above 
clients. The only thing I changed for Http11NioProtocol is to specify  
protocol="org.apache.coyote.http11.Http11NioProtocol” instead of 
protocol="org.apache.coyote.http11.Http11Nio2Protocol”. Also, I have another  
connector configured not checking client certificate. This one also works fine 
regardless of Http11NioProtocol or Http11Nio2Protocol being used.

Would you please help to identify if I have anything wrong in my configuration? 
 I tried to set the log level to fine. But I did not find anything useful. 
Please help.

8.5.31 fixes an error in this area that might be relevant. The release
vote for 8.5.31 is currently in progress. Details on the dev@ list. If
you could download the 8.5.31 release candidate and test against that,
that would be helpful.

If that doesn't work then we'll need the following (again with 8.5.31 so
we are testing the latest code):

Enable debug logging for the I/O layer:
org.apache.tomcat.util.net.level=FINE

Enable TLS debug logging for the client:
-Djavax.net.debug=all

Recreate the problem.

Provide us with:
- the logs for the 30s before the error and 5s after it
- the point in the logs where the error occurred

Thanks,

Mark

-
To unsubscribe, e-mail: 
users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: 
users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: 
users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: 
users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: 
users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: 
users-h...@tomcat.apache.org

Insert key-store implementation into Tomcat Connector

[Mark Boon]

In the Tomcat TLS Connecter configuration, there’s the trustManagerClassName 
that can be set to a Java implementation of the X509TrustManager interface. 
There’s also a configuration called keystoreFile from which it will read the 
certificate-key pair to set up the SSL connection. I was wondering if there’s 
also a way to configure a class that will provide the SSL certificate? My 
company would like to plug in their own mechanism to store and retrieve 
certificates, rather than the Java Key Store.

I have seen references to a keystoreProvider, but I have been unable to find 
anything that provides an example how that is to be used, so I’m not sure that 
serves for what I’m looking for.

Any pointer to how that could be accomplished would be highly appreciated.

Mark Boon

Information on sessionCacheSize !

[Utkarsh Dave]

Hello Team and Tomcat users,

I am trying to gather more information and the effect of parameter
"sessionCacheSize" in server.xml for a ssl connector.
I see this from the documentation "The number of SSL sessions to maintain
in the session cache."
If i do not add this parameter...my tomcat slows down and all the web
access becomes extremly slow within a couple of days.
This is because by default "0" size is assigned to this parameter which
means unlimited cached sessions.
So we added the parameter with the value of sessionCacheSize=1
What is the effect of 10k cached session on tomcat, can the problem reoccur
once 10k sessions are cached back.
I am planning to modify it to test this with a value of sessionCacheSize=1.
How can I test to come to a good value for sessioncachesize.

My product is using tomcat 7.0.81 (bio connector) with openjdk1.7.0.161 on
Linux RedHat 6.

-Thanks
Utkarsh

Problem finding native shared library (.so)

[Reynolds, Scott]

Hi,

I'm trying to deploy a webapp to Tomcat 8.0.39 on CentOS 7 x86_64 that depends 
on native shared libraries.  I can't install the shared libraries in a 
system-wide location because there are multiple applications/webapps being 
deployed to this system that use different versions of the same native shared 
libraries.  We're already running two instances of Tomcat to isolate the 
webapps from one another.  Here's what's going on:

Tomcat #1 (/opt/tomcatwx) - The webapp described below is able to find all 
necessary native shared libraries

bin/setenv.sh source's bin/setenv-wx.sh.

bin/setenv-wx.sh appends 
-Djava.library.path=/usr/lib64:/lib64:/lib:/usr/lib:/opt/tomcatwx/lib/gdal to 
JAVA_OPTS.

conf/catalina.properties appends 
,"${catalina.home}/lib/gdal","${catalina.home}/lib/gdal/*.jar" to common.loader.

lib/gdal contains the following files:
drwxr-xr-x. 2 tomcatwx root 4096 Apr 16 01:06 data
-rw-r--r--. 1 tomcatwx root   134785 Apr 16 00:59 gdal.jar
-rwxr-xr-x. 1 tomcatwx root86176 Apr 16 00:59 libgdalconstjni.so
-rwxr-xr-x. 1 tomcatwx root  1911144 Apr 16 00:59 libgdaljni.so
-rwxr-xr-x. 1 tomcatwx root 93499064 Apr 16 00:59 libgdal.so
-rwxr-xr-x. 1 tomcatwx root   222448 Apr 16 00:59 libgnmjni.so
-rwxr-xr-x. 1 tomcatwx root   968920 Apr 16 00:59 libogrjni.so
-rwxr-xr-x. 1 tomcatwx root   483536 Apr 16 00:59 libosrjni.so
-rw-r--r--. 1 tomcatwx root   335488 Apr 16 15:28 libproj.so
-rw-r--r--. 1 tomcatwx root 2021 Apr 16 01:06 ./lib/gdal/data/gdalicon.png
-rw-r--r--. 1 tomcatwx root19884 Apr 16 01:06 ./lib/gdal/data/gdalvrt.xsd
-rw-r--r--. 1 tomcatwx root   234839 Apr 16 01:06 ./lib/gdal/data/gdal_datum.csv
-rwxr-xr-x. 1 tomcatwx root  1911144 Apr 16 00:59 ./lib/gdal/libgdaljni.so
-rwxr-xr-x. 1 tomcatwx root86176 Apr 16 00:59 ./lib/gdal/libgdalconstjni.so
-rw-r--r--. 1 tomcatwx root   134785 Apr 16 00:59 ./lib/gdal/gdal.jar
-rwxr-xr-x. 1 tomcatwx root 93499064 Apr 16 00:59 ./lib/gdal/libgdal.so


Tomcat #2 (/srv/tomcat) - The webapp described below is unable to find a native 
shared library referenced by the JNI shared library.
Native library load failed.
java.lang.UnsatisfiedLinkError: /srv/tomcat/lib/3p/libgdaljni.so: 
libgdal.so.20: cannot open shared object file: No such file or directory

bin/setenv.sh source's bin/setenv-3p.sh.

bin/setenv-3-.sh appends -Djava.library.path=/srv/tomcat/lib/3p to JAVA_OPTS.

conf/catalina.properties appends 
,"${catalina.home}/lib/3p","${catalina.home}/lib/3p/*.jar" to common.loader.

lib/3p contains the following files:
drwxr-xr-x. 2 tomcat tomcat 4096 Dec  5 20:04 gdal
-rwxr-xr-x. 1 tomcat tomcat   128680 Dec  5 20:06 libgdalconstjni.so
-rwxr-xr-x. 1 tomcat tomcat  1982984 Dec  5 20:06 libgdaljni.so
lrwxrwxrwx. 1 tomcat tomcat   17 Dec  5 20:04 libgdal.so -> 
libgdal.so.20.3.101632
lrwxrwxrwx. 1 tomcat tomcat   17 Dec  5 20:04 libgdal.so.20 -> 
libgdal.so.20.3.101632
-rwxr-xr-x. 1 tomcat tomcat 93494552 Dec  5 20:04 libgdal.so.20.3.101632
-rwxr-xr-x. 1 tomcat tomcat   294464 Dec  5 20:06 libgnmjni.so
-rw-r--r--. 1 tomcat tomcat  3409872 Jul 15  2016 libjhdf5.so
-rw-r--r--. 1 tomcat tomcat  1268496 Jul 15  2016 libjhdf.so
-rwxr-xr-x. 1 tomcat tomcat  1039816 Dec  5 20:06 libogrjni.so
-rwxr-xr-x. 1 tomcat tomcat   554768 Dec  5 20:06 libosrjni.so
lrwxrwxrwx. 1 tomcat tomcat   17 Dec  5 19:35 libproj.so -> 
libproj.so.12.0.03
lrwxrwxrwx. 1 tomcat tomcat   17 Dec  5 19:35 libproj.so.12 -> 
libproj.so.12.0.03
-rwxr-xr-x. 1 tomcat tomcat  1871352 Dec  5 19:35 libproj.so.12.0.03
drwxr-xr-x. 2 tomcat tomcat 4096 Dec  5 19:35 proj

webapps/myapp/WEB-INF/lib contains the JNI jar:
-rw-r--r--. 1 tomcat tomcat 134646 May  1 14:03 
webapps/myapp/WEB-INF/lib/gdal-2.2.2.jar

which references libgdaljni.so
libgdaljni.so references libgdal.so
which references libgdal.so.20
which references libgdal.so.20.3.101632

What have I failed to do that is preventing Tomcat/Java from finding the fully 
resolved libgdal.so that is in the same directory where it found libgdaljni.so?

I tried eliminating the symbolic links by copying  libgdal.so.20.3.101632 to 
libgdal.so, without success.
I tried copying the gdal-2.2.2.jar to the lib/3p directory, without success.

Thanks in advance for any help.

Scott

Re: tomcat9 j_security_check request.getRequestURI() incorrect after POST

[Dirk Ooms]

apologies for the incomplete info. it is tomcat 9.0.6

i will try to set up a test case and get back to you.

dirk


On 1 May 2018 at 16:07, Mark Thomas  wrote:

> On 01/05/18 14:36, Dirk Ooms wrote:
> > Hello,
> >
> > i did an upgrade from tomcat5.5 to tomcat9 and i'm using
> j_security_check.
> >
> > in tomcat5.5 when a user was not logged in and he/she requested a url,
> the
> > login page was returned and after logging in the user was given the
> > requested resource. when i requested request.getRequestURI() in my code
> the
> > returned uri was correct for both GET and POST.
> >
> > in tomcat9 this is not the case anymore for POST (for GET still ok).
> when i
> > call request.getRequestURI() after the user is logged in, it returns
> > "chString" in my case, which is a part of the name of the first form
> field
> > ("searchString") of the original POST.
> >
> > any idea? am i missing something?
>
> The exact Tomcat 9 version.
>
> A test case that demonstrates the issue.
>
> Mark
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

Re: slow or timeout with client certificate and some http client against tomcat 8.5 with Nio2 OpenSSL implementation

[Rémy Maucherat]

On Tue, May 1, 2018 at 3:31 PM Mark Thomas  wrote:

> On 01/05/18 03:11, 旭东 胡 wrote:
> > Hi Mark,
> >
> > Unfortunately,  8.5.31 does not resolve my issue. You can find the
> catalina.out log by https://1drv.ms/u/s!Aii8T4l0bnqVlx0mqtHngJ_1OvRo.
> > From my client log the timeout occurs:
> > 1. between 15:03:48 and 15:04:48
> > 2. between 15:04:48 and 15:05:48
> > 3. between 15:05:49 and 15:06:49
> > 4. between 15:06:59 and 15:07:49
> > 5. between 15:07:59 and 15:08:49
> > 6. between 15:08:59 and 15:09:49
> >
> > The problematic port is 11443. Sorry there is a health checking, which I
> cannot turned off, on port 10443 adding a lot noise.
>
> OK.
>
> First of all, please ensure that the time on the client and server are
> synchronized. Give that the server log doesn't show the server starting
> until 15:04:00 the client and server look to be ~25 seconds out of sync.
>
> What I see in most of the connections is the TLS handshake completing
> and the I/O layer passing the socket to the protocol layer for
> processing. The socket is returned from the protocol layer with an
> instruction to close the socket.
>
> We need to see what is happening in the protocol layer. Please add the
> following to logging.properties, restart Tomcat 8.5.31 and repeat your
> test:
> org.apache.coyote.level=FINE
>
> Please also include the client logs this time.
>

Maybe test with JSSE as well instead of OpenSSL ?

Rémy

Re: tomcat9 j_security_check request.getRequestURI() incorrect after POST

[Mark Thomas]

On 01/05/18 14:36, Dirk Ooms wrote:
> Hello,
> 
> i did an upgrade from tomcat5.5 to tomcat9 and i'm using j_security_check.
> 
> in tomcat5.5 when a user was not logged in and he/she requested a url, the
> login page was returned and after logging in the user was given the
> requested resource. when i requested request.getRequestURI() in my code the
> returned uri was correct for both GET and POST.
> 
> in tomcat9 this is not the case anymore for POST (for GET still ok). when i
> call request.getRequestURI() after the user is logged in, it returns
> "chString" in my case, which is a part of the name of the first form field
> ("searchString") of the original POST.
> 
> any idea? am i missing something?

The exact Tomcat 9 version.

A test case that demonstrates the issue.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

tomcat9 j_security_check request.getRequestURI() incorrect after POST

[Dirk Ooms]

Hello,

i did an upgrade from tomcat5.5 to tomcat9 and i'm using j_security_check.

in tomcat5.5 when a user was not logged in and he/she requested a url, the
login page was returned and after logging in the user was given the
requested resource. when i requested request.getRequestURI() in my code the
returned uri was correct for both GET and POST.

in tomcat9 this is not the case anymore for POST (for GET still ok). when i
call request.getRequestURI() after the user is logged in, it returns
"chString" in my case, which is a part of the name of the first form field
("searchString") of the original POST.

any idea? am i missing something?

thanks,
dirk

Re: slow or timeout with client certificate and some http client against tomcat 8.5 with Nio2 OpenSSL implementation

[Mark Thomas]

On 01/05/18 03:11, 旭东 胡 wrote:
> Hi Mark,
> 
> Unfortunately,  8.5.31 does not resolve my issue. You can find the 
> catalina.out log by https://1drv.ms/u/s!Aii8T4l0bnqVlx0mqtHngJ_1OvRo. 
> From my client log the timeout occurs:
> 1. between 15:03:48 and 15:04:48
> 2. between 15:04:48 and 15:05:48
> 3. between 15:05:49 and 15:06:49
> 4. between 15:06:59 and 15:07:49
> 5. between 15:07:59 and 15:08:49
> 6. between 15:08:59 and 15:09:49
> 
> The problematic port is 11443. Sorry there is a health checking, which I 
> cannot turned off, on port 10443 adding a lot noise.

OK.

First of all, please ensure that the time on the client and server are
synchronized. Give that the server log doesn't show the server starting
until 15:04:00 the client and server look to be ~25 seconds out of sync.

What I see in most of the connections is the TLS handshake completing
and the I/O layer passing the socket to the protocol layer for
processing. The socket is returned from the protocol layer with an
instruction to close the socket.

We need to see what is happening in the protocol layer. Please add the
following to logging.properties, restart Tomcat 8.5.31 and repeat your test:
org.apache.coyote.level=FINE

Please also include the client logs this time.

Thanks,

Mark


> 
> Thanks,
> Hugh
> 
>> On Apr 30, 2018, at 5:08 AM, Mark Thomas  wrote:
>>
>> On 30/04/18 01:48, ** * wrote:
>>> Hi,
>>>
>>> I met a weird issue during setting up tomcat 8.5 with Http11Nio2Protocol 
>>> connector and OpenSSLImplementation. The issue is that a request would be 
>>> timeout using apache HttpClient and client certificate after serval 
>>> previous requests. It also happens with RestAssured and SoapUI. Please note 
>>> it works fine for first several requests and then failed with timeout.
>>>
>>> However, this issue is not observed when JMeter, tried both JAVA and 
>>> non-JAVA implementation, and insomnia REST client being used. I used a 
>>> static page to rule out application factors. Also Http11NioProtocol works 
>>> fine for all above clients. The only thing I changed for Http11NioProtocol 
>>> is to specify  protocol="org.apache.coyote.http11.Http11NioProtocol” 
>>> instead of  protocol="org.apache.coyote.http11.Http11Nio2Protocol”. Also, I 
>>> have another  connector configured not checking client certificate. This 
>>> one also works fine regardless of Http11NioProtocol or Http11Nio2Protocol 
>>> being used.
>>>
>>> Would you please help to identify if I have anything wrong in my 
>>> configuration?  I tried to set the log level to fine. But I did not find 
>>> anything useful. Please help.
>>
>> 8.5.31 fixes an error in this area that might be relevant. The release
>> vote for 8.5.31 is currently in progress. Details on the dev@ list. If
>> you could download the 8.5.31 release candidate and test against that,
>> that would be helpful.
>>
>> If that doesn't work then we'll need the following (again with 8.5.31 so
>> we are testing the latest code):
>>
>> Enable debug logging for the I/O layer:
>> org.apache.tomcat.util.net.level=FINE
>>
>> Enable TLS debug logging for the client:
>> -Djavax.net.debug=all
>>
>> Recreate the problem.
>>
>> Provide us with:
>> - the logs for the 30s before the error and 5s after it
>> - the point in the logs where the error occurred
>>
>> Thanks,
>>
>> Mark
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
> 
> 
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

ApacheCon North America 2018 schedule is now live.

[Rich Bowen]

Dear Apache Enthusiast,

We are pleased to announce our schedule for ApacheCon North America 
2018. ApacheCon will be held September 23-27 at the Montreal Marriott 
Chateau Champlain in Montreal, Canada.


Registration is open! The early bird rate of $575 lasts until July 21, 
at which time it goes up to $800. And the room block at the Marriott 
($225 CAD per night, including wifi) closes on August 24th.


We will be featuring more than 100 sessions on Apache projects. The 
schedule is now online at https://apachecon.com/acna18/


The schedule includes full tracks of content from Cloudstack[1], 
Tomcat[2], and our GeoSpatial community[3].


We will have 4 keynote speakers, two of whom are Apache members, and two 
from the wider community.


On Tuesday, Apache member and former board member Cliff Schmidt will be 
speaking about how Amplio uses technology to educate and improve the 
quality of life of people living in very difficult parts of the 
world[4]. And Apache Fineract VP Myrle Krantz will speak about how Open 
Source banking is helping the global fight against poverty[5].


Then, on Wednesday, we’ll hear from Bridget Kromhout, Principal Cloud 
Developer Advocate from Microsoft, about the really hard problem in 
software - the people[6]. And Euan McLeod, ‎VP VIPER at ‎Comcast will 
show us the many ways that Apache software delivers your favorite shows 
to your living room[7].


ApacheCon will also feature old favorites like the Lightning Talks, the 
Hackathon (running the duration of the event), PGP key signing, and lots 
of hallway-track time to get to know your project community better.


Follow us on Twitter, @ApacheCon, and join the disc...@apachecon.com 
mailing list (send email to discuss-subscr...@apachecon.com) to stay up 
to date with developments. And if your company wants to sponsor this 
event, get in touch at h...@apachecon.com for opportunities that are 
still available.


See you in Montreal!

Rich Bowen
VP Conferences, The Apache Software Foundation
h...@apachecon.com
@ApacheCon

[1] http://cloudstackcollab.org/
[2] http://tomcat.apache.org/conference.html
[3] http://apachecon.dukecon.org/acna/2018/#/schedule?search=geospatial
[4] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/df977fd305a31b903
[5] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/22c6c30412a3828d6
[6] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/fbbb2384fa91ebc6b
[7] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/88d50c3613852c2de


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

TC 8.5.27 clustering doesn't work as described(?) and expected

[Stefan Hall]

Hi,

I've probably read everything in the last few days and tried even more, 
but it won't work. I need your experience.


First, my expectations of the Tomcat Cluster
 1. session failover when a Tomcat dies (simulated via kill -9)
 2. session failover when I close a Tomcat (via shutdown.sh or 
simulated via kill)


Point 1 works, but point 2 does not work despite className="org.apache.catalina.ha.session.DeltaManager" 
expireSessionsOnShutdown="false" ...>


It seems as if all active sessions of the Tomcat are expired during the 
shutdown process - in the whole cluster. This contradicts in my opinion 
the parameter expireSessionsOnShutdown="false" and its explanation. I 
can also set the parameter to true, no change.


Hope you can help me.
Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org