HttpClient 4.5 does not use the WebSphere 7 truststore

2017-05-18 Thread Jonathan Barbero 
Hi,

 I'm using the version 4.5 of HttpClient on a servlet in a WAS 7 calling to
another WAS 7.
When I test the call with http protocol everything works. But when I try
with https the call fails because "Chaining certificate error", the
certificate of the CA is not recognized as trusted.

 The CA certificate is in the WAS truststore. When we call in another app
with a JAX-WS client over https to the same endpoint it works, so it get
the CA certificate from the truststore. Also, I capture in the WAS log that
when it starts the app server loads the CA certificate as trusted


[5/18/17 18:17:39:867 ART]  CSIServerRI   A   JSAS0008I: Server
request interceptor registered.
[5/18/17 18:17:39:878 ART]  SecurityCompo A   JSAS0009I: IOR
interceptor registered.
[5/18/17 18:17:40:732 ART]  SystemOut O adding as trusted cert:
[5/18/17 18:17:40:734 ART]  SystemOut O   Subject: CN=CABNA,
DC=cc, DC=bna, DC=net
[5/18/17 18:17:40:737 ART]  SystemOut O   Issuer:  CN=CABNA,
DC=cc, DC=bna, DC=net
[5/18/17 18:17:40:744 ART]  SystemOut O   Algorithm: RSA;
Serial number: 0x476da8f2b43899b24dfe7a94e66a1b7f
[5/18/17 18:17:40:747 ART]  SystemOut O   Valid from Mon May 09
12:38:13 ART 2016 until Sat May 09 12:48:13 ART 2026
[5/18/17 18:17:40:748 ART]  SystemOut O


But when I call my servlet and this one tries to call with an HttpClient to
the other WAS over https, I captured that it does not load the same
truststore of the WAS


[5/18/17 18:22:36:965 ART] 0028 SystemOut O keyStore is:*
/opt/IBM/WebSphere/AppServer/java/jre/lib/security/cacerts*
[5/18/17 18:22:36:965 ART] 0028 SystemOut O keyStore type is: jks
[5/18/17 18:22:36:965 ART] 0028 SystemOut O keyStore provider is:
[5/18/17 18:22:36:965 ART] 0028 SystemOut O init keystore
[5/18/17 18:22:37:237 ART] 000d SystemOut O Finalizer thread,
called close()
[5/18/17 18:22:37:237 ART] 000d SystemOut O Finalizer thread,
called closeInternal(true)
[5/18/17 18:22:37:237 ART] 000d SystemOut O Finalizer thread,
called close()
[5/18/17 18:22:37:237 ART] 000d SystemOut O Finalizer thread,
called closeInternal(true)
[5/18/17 18:22:37:248 ART] 0028 SystemOut O SSLContextImpl:  Using
X509ExtendedKeyManager com.ibm.jsse2.hd
[5/18/17 18:22:37:250 ART] 0028 SystemOut O trustStore is:
*/opt/IBM/WebSphere/AppServer/java/jre/lib/security/cacerts*
[5/18/17 18:22:37:250 ART] 0028 SystemOut O trustStore type is: jks
[5/18/17 18:22:37:250 ART] 0028 SystemOut O trustStore provider is:
[5/18/17 18:22:37:250 ART] 0028 SystemOut O init truststore
[5/18/17 18:22:37:253 ART] 0028 SystemOut O adding as trusted cert:
[5/18/17 18:22:37:253 ART] 0028 SystemOut O   Subject: CN=Certum
Trusted Network CA, OU=Certum Certification Authority, O=Unizeto
Technologies S.A., C=PL
[5/18/17 18:22:37:254 ART] 0028 SystemOut O   Issuer:  CN=Certum
Trusted Network CA, OU=Certum Certification Authority, O=Unizeto
Technologies S.A., C=PL
[5/18/17 18:22:37:254 ART] 0028 SystemOut O   Algorithm: RSA;
Serial number: 0x444c0
[5/18/17 18:22:37:254 ART] 0028 SystemOut O   Valid from Wed Oct 22
10:07:37 ARST 2008 until Mon Dec 31 09:07:37 ART 2029
[5/18/17 18:22:37:254 ART] 0028 SystemOut O
.
.
... and so on


seems like the HttpClient loads the certificates in cacerts of the JVM only.

So, I modified the creation of the HttpClient and added  useSystemProperties()
 because I read that this might take the truststore of the WAS, but didn't
work. Used a Basic connection manager and a Pooled one, but no difference.

Also tried to modify the certificate validation, using an strategy that
does nothing, but the verification fails anyway. The strategy that
validates nothing is never called.
The certificate validation failure appears when the HttpClient uses the IBM
socket (trace at the bottom of the email).

A couple of years ago I used a version of Spring WS with HttpClient 4.1 or
4.2 version and didn't have this problem.

Any help or tip is really welcome.

Regards,
 Jonathan.



Trace of the certificate verification:

[5/18/17 18:22:46:070 ART] 0028 SystemOut O ***
[5/18/17 18:22:46:174 ART] 0028 SystemOut O %% Invalidated:
 [Session-8, SSL_RSA_WITH_AES_128_CBC_SHA]
[5/18/17 18:22:46:174 ART] 0028 SystemOut O WebContainer : 0, SEND
TLSv1 ALERT:  fatal, description = certificate_unknown
[5/18/17 18:22:46:174 ART] 0028 SystemOut O WebContainer : 0,
WRITE: TLSv1 Alert, length = 2
[5/18/17 18:22:46:175 ART] 0028 SystemOut O WebContainer : 0,
called closeSocket()
[5/18/17 18:22:46:175 ART] 0028 SystemOut O WebContainer : 0,
handling exception: javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.h:
PKIX path building failed: java.security.cert.CertPathBuilderException:
PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause

Re: Async client with self signed certificate

2017-05-18 Thread Gary Gregory 
You can remove most of this boilerplate if use use the SslContextBuilder
class.

Gary

On May 18, 2017 11:48 AM, "Joan Balagueró" 
wrote:

> Hello,
>
>
>
> I’ve been using SSL with client authentication with signed certificates in
> async http client 4.1, with no problem.
>
>
>
> My code is:
>
>
>
> FileInputStream  fKeyStore = new FileInputStream(new
> File(keyStoreLocation));
>
> KeyStore keyStore = KeyStore.getInstance(keyStoreType);
>
> keyStore.load(fKeyStore, keyStorePassword.toCharArray());
>
>
>
> KeyManagerFactory kmfactory =
> KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
>
> kmfactory.init(keyStore, keyStorePassword.toCharArray());
>
> KeyManager[] keyManagers = kmfactory.getKeyManagers();
>
>
>
> TrustManagerFactory tmf =
> TrustManagerFactory.getInstance(TrustManagerFactory.
> getDefaultAlgorithm());
>
> tmf.init(keyStore);
>
>
>
> SSLContext sslContext = SSLContexts.custom().build();
>
> sslContext.init(keyManagers, tmf.getTrustManagers(), null);
>
>
>
> return (new SSLIOSessionStrategy(sslContext, new String[] { "TLSv1" },
> null,
> SSLIOSessionStrategy.getDefaultHostnameVerifier()));
>
>
>
>
>
> But now I have an installation with ssl and client authentication but with
> a
> self-signed certificate. Using the previous code I get the following error
> (I suppose because it doesn’t find the CA certificate):
>
> Caused by: sun.security.validator.ValidatorException: PKIX path building
> failed: sun.security.provider.certpath.SunCertPathBuilderException: unable
> to find valid certification path to requested target
>
>
>
> Can anyone help me with this? How should I modify the previous code to have
> this working? I’ve tried some alternatives but none of them worked.
>
>
>
> Thanks in advance.
>
>
>
> Joan.
>
>
>
>
>
>
>
>
>
>

Re: Async client with self signed certificate

2017-05-18 Thread Hassan Khan 
Hi,

This is a issue with the CA certs... SSL handshake is failing...

if java turn on ssl debug... you will see the error in detail...

But if you have added the cacert to the java cacert files.. then java
should recognize the self signed cert..

This is not a code issue.. it more to do with cert that is the point i am
trying to make... May be the file Store does not have the self signed
certificate added to it...

Hope it helps

Thanks
Hassan



On Thu, May 18, 2017 at 2:48 PM, Joan Balagueró <
joan.balagu...@grupoventus.com> wrote:

> Hello,
>
>
>
> I’ve been using SSL with client authentication with signed certificates in
> async http client 4.1, with no problem.
>
>
>
> My code is:
>
>
>
> FileInputStream  fKeyStore = new FileInputStream(new
> File(keyStoreLocation));
>
> KeyStore keyStore = KeyStore.getInstance(keyStoreType);
>
> keyStore.load(fKeyStore, keyStorePassword.toCharArray());
>
>
>
> KeyManagerFactory kmfactory =
> KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
>
> kmfactory.init(keyStore, keyStorePassword.toCharArray());
>
> KeyManager[] keyManagers = kmfactory.getKeyManagers();
>
>
>
> TrustManagerFactory tmf =
> TrustManagerFactory.getInstance(TrustManagerFactory.
> getDefaultAlgorithm());
>
> tmf.init(keyStore);
>
>
>
> SSLContext sslContext = SSLContexts.custom().build();
>
> sslContext.init(keyManagers, tmf.getTrustManagers(), null);
>
>
>
> return (new SSLIOSessionStrategy(sslContext, new String[] { "TLSv1" },
> null,
> SSLIOSessionStrategy.getDefaultHostnameVerifier()));
>
>
>
>
>
> But now I have an installation with ssl and client authentication but with
> a
> self-signed certificate. Using the previous code I get the following error
> (I suppose because it doesn’t find the CA certificate):
>
> Caused by: sun.security.validator.ValidatorException: PKIX path building
> failed: sun.security.provider.certpath.SunCertPathBuilderException: unable
> to find valid certification path to requested target
>
>
>
> Can anyone help me with this? How should I modify the previous code to have
> this working? I’ve tried some alternatives but none of them worked.
>
>
>
> Thanks in advance.
>
>
>
> Joan.
>
>
>
>
>
>
>
>
>
>


-- 
Hassan Khan

Async client with self signed certificate

2017-05-18 Thread Joan Balagueró 
Hello,

 

I’ve been using SSL with client authentication with signed certificates in
async http client 4.1, with no problem.

 

My code is:

 

FileInputStream  fKeyStore = new FileInputStream(new
File(keyStoreLocation));

KeyStore keyStore = KeyStore.getInstance(keyStoreType);

keyStore.load(fKeyStore, keyStorePassword.toCharArray());

 

KeyManagerFactory kmfactory =
KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());

kmfactory.init(keyStore, keyStorePassword.toCharArray());

KeyManager[] keyManagers = kmfactory.getKeyManagers();

 

TrustManagerFactory tmf =
TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());

tmf.init(keyStore);

 

SSLContext sslContext = SSLContexts.custom().build();

sslContext.init(keyManagers, tmf.getTrustManagers(), null);

 

return (new SSLIOSessionStrategy(sslContext, new String[] { "TLSv1" }, null,
SSLIOSessionStrategy.getDefaultHostnameVerifier()));

 

 

But now I have an installation with ssl and client authentication but with a
self-signed certificate. Using the previous code I get the following error
(I suppose because it doesn’t find the CA certificate):

Caused by: sun.security.validator.ValidatorException: PKIX path building
failed: sun.security.provider.certpath.SunCertPathBuilderException: unable
to find valid certification path to requested target

 

Can anyone help me with this? How should I modify the previous code to have
this working? I’ve tried some alternatives but none of them worked.

 

Thanks in advance.

 

Joan.

Re: HttpCacheStorage#getEntry called twice on cache hit

2017-05-18 Thread Leandro Nunes 
Hi,

I created this (https://github.com/apache/httpcomponents-client/pull/77) pull 
request with a simple possible solution to this problem. It would be awesome if 
you could please take a look and validate whether the proposed fix valid or 
not. I’m more than happy to change whatever you think is necessary and reapply 
the fix on other branches after that.

Thanks,
Leandro

On May 17, 2017, at 11:59 AM, Leandro Nunes 
mailto:a-lnu...@hotels.com>> wrote:

Thanks Oleg and Jon,

Oleg,
I didn’t see a problem with the implementation of the RFC. The result I get 
back is the one I was expecting (regarding actually calling the origin for the 
entity or returning the cached instance - without making the request to the 
origin). The problem has to do with calling the datastore twice (which can pose 
as a problem specially when using an off-heap datastore).

Jon,
I created a gist where you can see the behaviour I’m talking about 
(https://gist.github.com/leandronunes85/c8f68ca556ac4121bdd956b2ae29c19f). In 
this example I’m using a facebook static resource with “Cache-Control: 
public,max-age=31536000,immutable”. In my exact use case I get back this 
header: "Cache-Control: max-age=600, public”. In both cases the behaviour is 
the same. The output from running the application in the gist is:
op=getEntry, key='https://www.facebook.com:443/rsrc.php/v3/y4/r/gf6iHxsw8zm.png'
op=getEntry, key='https://www.facebook.com:443/rsrc.php/v3/y4/r/gf6iHxsw8zm.png'
op=getEntry, key='https://www.facebook.com:443/rsrc.php/v3/y4/r/gf6iHxsw8zm.png'
op=getEntry, key='https://www.facebook.com:443/rsrc.php/v3/y4/r/gf6iHxsw8zm.png'
op=putEntry, key='https://www.facebook.com:443/rsrc.php/v3/y4/r/gf6iHxsw8zm.png'
CACHE_MISS - HTTP/1.1 200 OK - Cache-Control: public,max-age=31536000,immutable
op=getEntry, key='https://www.facebook.com:443/rsrc.php/v3/y4/r/gf6iHxsw8zm.png'
op=getEntry, key='https://www.facebook.com:443/rsrc.php/v3/y4/r/gf6iHxsw8zm.png'
CACHE_HIT - HTTP/1.1 200 OK - Cache-Control: public,max-age=31536000,immutable
op=getEntry, key='https://www.facebook.com:443/rsrc.php/v3/y4/r/gf6iHxsw8zm.png'
op=getEntry, key='https://www.facebook.com:443/rsrc.php/v3/y4/r/gf6iHxsw8zm.png'
CACHE_HIT - HTTP/1.1 200 OK - Cache-Control: public,max-age=31536000,immutable

As you can see the datastore is being asked for the entity twice on cache hit 
(and 4 times (!) for cache misses).

Thanks for any help you can provide,
Leandro

On May 17, 2017, at 12:39 AM, Jon Moore 
mailto:j...@apache.org>> wrote:

Yes, unfortunately I haven't had time to work on the caching module for
several years, so I don't remember all the ins and outs of the
implementation. However, Leandro, perhaps you could post the headers for
the original request, the (cached) response, and then the subsequent
request that gets a cache hit? If so, I can possibly help explain the
behavior.

Jon

On Tue, May 16, 2017 at 2:23 PM, Oleg Kalnichevski 
mailto:ol...@apache.org>> wrote:

On Tue, 2017-05-16 at 11:42 +, Leandro Nunes wrote:
I just noticed the caching implementation is calling getEntry method
twice when the entry is cached. I did some debugging and it looks
like the first one is related with flushing invalid entries and the
second call is related with actually serving the request. My question
is wether this is the expected behaviour or is something that has
been overlooked and can/should be addressed?

Thanks,
Leandro

Leandro


HttpClient cache module desperately needs some attention. The best
course of action should be to consult the RFC, fix the behavior if
wrong, raise a PR at Github.

Oleg


-
To unsubscribe, e-mail: 
httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: 
httpclient-users-h...@hc.apache.org

Re: Upgrading from Httpclient 3.1 to 4.5 - localhost:443 not responding

2017-05-18 Thread Hassan Khan 
You are right..  Thanks.. It was kinda of a wrong question to ask ...

On Thu, May 18, 2017 at 3:35 AM, Oleg Kalnichevski  wrote:

> On Wed, 2017-05-17 at 12:55 -0400, Hassan Khan wrote:
> > Thank oleg for the tip..
> >
> > I did not change the connector till now.. but with APR itself I
> > starting
> > using the prod CA certificate that our company has... instead of the
> > self
> > signed certificate...
> >
> > With httpClient 3.1 all communication work fine.
> >
>
> As I have already explained earlier. HC 3.x does _not_ do any hostname
> validation. It just does not.
>
> > But when I upgraded prod to use the new code having httpclient
> > 4.5 I
> > get this exception in SSL handshake...
> > Certificate for XVT doesn't match any of the subject alternative
> > names:
> > ABC, GFD]
> >
> > So looks like I need to turn off the hostname verification in the
> > code or
> > update the Com[any certificate to have CN populated with the values.
> >
>
> No, you should rather make sure that the hostname and the cert
> presented by the server match.
>
>
> > I wanted to know what brought the need to have CN in every
> > Certificate
> > populated gong forward?
> >
>
> Eh, like, CN being mandatory for a cert, should be a good reason,
> should it not?
>
> Oleg
>
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
>


-- 
Hassan Khan

How replace the default HTTP URL handler in Java with one that uses the Apache HTTPClient?

2017-05-18 Thread Mats Eklund 
Hi,

I am thinking of replacing the default HTTP connection handler in my Java web 
applications with one that uses Apache HTTPClient. The reason is that I want to 
leverage response caching of outbound HTTP requests made from my web 
application. The requests are made via various libraries, all using the 
standard java.net.URL class to access resources on the web.

Does the HTTP Components framework contain a ready to use solution for this, or 
are there some guidelines available for how to best go about it. I guess what I 
am looking for could be a common scenario! I found some possible directions 
here:

http://mjremijan.blogspot.co.at/2012/02/create-your-own-java-url-handlers.html

http://stackoverflow.com/questions/27993995/creating-httpurlconnection-for-urlstreamhandlerfactory-getting-error-protocol/28101085#28101085

But maybe there is already a ready to use component available, that could be 
simply dropped along with a configuration file for the caching behavior into 
the application's libraries folder?

Thanks so much in advance for any advice!

Mats

Re: Upgrading from Httpclient 3.1 to 4.5 - localhost:443 not responding

2017-05-18 Thread Oleg Kalnichevski 
On Wed, 2017-05-17 at 12:55 -0400, Hassan Khan wrote:
> Thank oleg for the tip..
> 
> I did not change the connector till now.. but with APR itself I
> starting
> using the prod CA certificate that our company has... instead of the
> self
> signed certificate...
> 
> With httpClient 3.1 all communication work fine.
> 

As I have already explained earlier. HC 3.x does _not_ do any hostname
validation. It just does not. 

> But when I upgraded prod to use the new code having httpclient
> 4.5 I
> get this exception in SSL handshake...
> Certificate for XVT doesn't match any of the subject alternative
> names:
> ABC, GFD]
> 
> So looks like I need to turn off the hostname verification in the
> code or
> update the Com[any certificate to have CN populated with the values.
> 

No, you should rather make sure that the hostname and the cert
presented by the server match.


> I wanted to know what brought the need to have CN in every
> Certificate
> populated gong forward?
> 

Eh, like, CN being mandatory for a cert, should be a good reason,
should it not? 

Oleg

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org