Ok, if you want the simplest thing that will work try the code below. This
time I’ve tested it. However, keep in mind this code removes much of the
benefit of SSL between the client and Knox due to the use of
TrustSelfSignedStrategy and NoopHostnameVerifier. If you are using this in
production there are a few different routes to go. For example if you are
using CA signed certs much of the SSL setup code below isn’t even required.
Can you provide more context about what you are actually trying to accomplish?
import org.apache.http.HttpEntity;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.util.EntityUtils;
import javax.net.ssl.SSLContext;
public class HttpClientSslNoVerifySslSample {
public static void main( String[] args ) throws Exception {
SSLContext sslContext = SSLContextBuilder.create()
.loadTrustMaterial( new TrustSelfSignedStrategy() ) // *** Trust self
signed certs. ***
.build();
SSLConnectionSocketFactory sslFactory = new SSLConnectionSocketFactory(
sslContext );
CloseableHttpClient client = HttpClients.custom()
.setSSLSocketFactory( sslFactory )
.setSSLHostnameVerifier( new NoopHostnameVerifier() ) // *** Allow all
host names. ***
.build();
HttpClientContext cliContext = HttpClientContext.create();
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(
new AuthScope( AuthScope.ANY_HOST, AuthScope.ANY_PORT ),
new UsernamePasswordCredentials( "guest", "guest-password" ) );
cliContext.setCredentialsProvider( credentialsProvider );
HttpGet method = new HttpGet(
"https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY"; );
CloseableHttpResponse response = client.execute( method, cliContext );
HttpEntity entity = response.getEntity();
System.out.println( EntityUtils.toString( entity ) );
response.close();
client.close();
}
}
From: Hafiz Mujadid <[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Wednesday, July 8, 2015 at 4:53 AM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: Apache Knox Web API
Hi Kevin!
I tried this code and got following exception
Error: keytool error: java.io.IOException: Keystore was tampered with, or
password was incorrect
java.io.IOException: Keystore was tampered with, or password was incorrect
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772)
at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55)
I regenerated password for key store and replaced
trustStore.load(stream, "wrong".toCharArray())
to
trustStore.load(stream, "changeit".toCharArray())
but still it's not working.
On Wed, Jul 8, 2015 at 1:46 AM, Kevin Minder
<[email protected]<mailto:[email protected]>> wrote:
Take a look at this below. This is a bit of a mod of an existing sample I had
laying around so don’t take it as tested.
import org.apache.http.HttpEntity;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.conn.ssl.AllowAllHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLContexts;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
import javax.net.ssl.SSLContext;
import java.io.File;
import java.io.FileInputStream;
import java.security.KeyStore;
public class HttpClientSslTest {
public static void main( String[] args ) throws Exception {
KeyStore trustStore = KeyStore.getInstance( KeyStore.getDefaultType() );
FileInputStream stream = new FileInputStream( new File( "gateway.jks" ) );
trustStore.load( stream, "wrong".toCharArray() );
stream.close();
SSLContext sslContext = SSLContexts.custom()
.loadTrustMaterial( trustStore, new TrustSelfSignedStrategy() ) // ***
Trust self signed certs. ***
.build();
SSLConnectionSocketFactory sslFactory = new SSLConnectionSocketFactory(
sslContext );
CloseableHttpClient client = HttpClients.custom()
.setSSLSocketFactory( sslFactory )
.setHostnameVerifier( new AllowAllHostnameVerifier() ) // *** Trust all
host names. ***
.build();
HttpClientContext cliContext = HttpClientContext.create();
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(
new AuthScope( AuthScope.ANY_HOST, AuthScope.ANY_PORT ),
new UsernamePasswordCredentials( "guest", "guest-password" ) );
cliContext.setCredentialsProvider( credentialsProvider );
HttpGet method = new HttpGet(
"https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY"; );
CloseableHttpResponse response = client.execute( method, cliContext );
HttpEntity entity = response.getEntity();
System.out.println( EntityUtils.toString( entity ) );
response.close();
client.close();
}
}
From: Hafiz Mujadid <[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Tuesday, July 7, 2015 at 4:05 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: Apache Knox Web API
Hi larry!
As suggested by you, I tried to use knox rest api using Apache HttpClient
here is my code
val provider = new BasicCredentialsProvider()
val credentials = new UsernamePasswordCredentials("admin", "12345")
provider.setCredentials(AuthScope.ANY, credentials)
val client =
HttpClientBuilder.create().setDefaultCredentialsProvider(provider) .build()
val response = client.execute(new
HttpGet("https://localhost:8443/gateway/sample/webhdfs/v1?op=LISTSTATUS";))
val statusCode = response.getStatusLine.getStatusCode
val input = response.getEntity().getContent()
if (statusCode == HttpStatus.SC_OK)
println("ok")
but I am getting following SSL related exception.
Exception in thread "main" javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)
Any suggestion?
On Mon, Jul 6, 2015 at 10:34 PM, Hafiz Mujadid
<[email protected]<mailto:[email protected]>> wrote:
thanks for your help .:)
On Mon, Jul 6, 2015 at 10:05 PM, larry mccay
<[email protected]<mailto:[email protected]>> wrote:
As I mentioned, you can dig into the source of the gateway-shell classes -
which are used when scripting with groovy.
Here is a link to an hdfs Get request:
https://github.com/apache/knox/blob/master/gateway-shell/src/main/java/org/apache/hadoop/gateway/shell/hdfs/Get.java#L32
Going to the HttpClient level is like going to bare metal - it provides you
greatest level of control but you will need to build abstractions around its
use in order to avoid lots of redundant boilerplate code. Which is why we have
provided such classes for the scripting.
You can also look at the DefaultDispatch code as an example - it is a bit more
complicated since it covers more general usecases but you may glean some
insights from it.
Otherwise, google for examples of "Apache HttpClient REST basic authentication"
and see what you find.
Hope this is useful for you!
On Sun, Jul 5, 2015 at 11:40 AM, Hafiz Mujadid
<[email protected]<mailto:[email protected]>> wrote:
Hi Larry!
Can you provide the link to samples using httclient on github etc.?
Thanks
On Sat, Jul 4, 2015 at 9:40 PM, larry mccay
<[email protected]<mailto:[email protected]>> wrote:
Then you will want to consider the Client library from the first link.
You can look in the {GATEWAY_HOME}/samples directory for examples of it's use.
The groovy scripts are a great way to do it or you can use the underlying java
classes that groovy uses.
The latter will require you to dig into the source a bit more to see how to use
them.
You can also use Apache HttpClient and there are samples of that as well.
On Sat, Jul 4, 2015 at 12:04 PM, Aneela Saleem
<[email protected]<mailto:[email protected]>> wrote:
Thanks Larry.
Actually I need some client API like java so that I authenticate / authorize
my users programmatically through Knox.
On Sat, Jul 4, 2015 at 8:50 PM, larry mccay
<[email protected]<mailto:[email protected]>> wrote:
Hi Aneela -
I assume that you mean that you would like to add support for a Hadoop API that
Knox currently lacks.
My suggestion is that you find one that your organization or your personal
interests require.
There are lots of Jira's filed for bug fixes and other features/enhancements as
well.
Feel free to start a discussion regarding any contribution that you would like
to make.
As far as the links that you referenced:
1. The first is a client library for scripting interactions with Hadoop
services through Knox - there are some really interesting and powerful
capabilities there.
2. The second is actually pointing to a section the dev guide that needs to be
completed. We have what we call Gateway Services in the kernel of the Knox
server that provide implementations for core server interfaces - crypto, SSL,
credential aliasing, etc. I don't think that you want to work in that space. If
you want to work on adding new API support for services then you should refer
to the Services section -
https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
Note that the link that I provided above is for the 0.6.0 dev guide. There is a
new configuration driven way to add API support to Knox that was added in the
0.6.0 release.
Thanks for your interest in contributing to Apache Knox!
--larry
On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem
<[email protected]<mailto:[email protected]>> wrote:
Hi Everyone,
I'm going to start development for Hadoop security through Apache Knox. Can
anyone please suggest me some good API for Knox.
So far i have found following:
https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
--
Regards: HAFIZ MUJADID
--
Regards: HAFIZ MUJADID
--
Regards: HAFIZ MUJADID
--
Regards: HAFIZ MUJADID
