[jira] [Commented] (CONNECTORS-1629) Support Solr Kerberos Authentication
[ https://issues.apache.org/jira/browse/CONNECTORS-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007094#comment-17007094 ] Jörn Franke commented on CONNECTORS-1629: - Thanks for the quick feedback. I can do this. Is Github pull request for this adequate or should I use the Apache one? About the ModifiedSolrClient - do I understand you correctly that you would prefer to make the ModifiedSolrClient working in this setting as well? Ie by creating a new ModifiedSolrClientKerberos and ModifiedLBSolrClientKerberos (not touching the ones already in Manifold)? I can look at this, but I wonder if this would still be needed as I did not observe any errors. Maybe the multipart bit is fixed in higher Solr versions? > Support Solr Kerberos Authentication > > > Key: CONNECTORS-1629 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1629 > Project: ManifoldCF > Issue Type: Improvement > Components: Solr 7.x component >Affects Versions: ManifoldCF 2.14 >Reporter: Jörn Franke >Priority: Major > > Several enterprise deployments of Solr are leveraging SolrCloud Kerberos > authentication. > The integration seems to be rather simple and the goal of this Jira is to > evaluate the possential needed step to eventually contribute the Kerberos > integration to the ManifoldCF project. > The following steps would be needed: > * One can pass the JVM parameter java.security.auth.login.config to the > ManifoldCF JVM using -Djava.security.auth.login.config=/path/to/jaas.confg in > which Kerberos authentication details, such as keytab and principal that has > the right access to Solr is configured > * A small adaption to the SolrCloudClient that is used within Manifold needs > to be done to enable Kerberos authentication: > HttpClientUtil.setConfigurer(new Krb5HttpClientConfigurer()); > Should this be integrated in Manifold, one may want to consider one input > field in the configuration in the UI where one can select / flow which user > defined in the Jaas conf (you can define multiple one) should be chosen. By > default one may simply select "client" or "SolrJClient" if Jaas.conf is > present in the System properties. This does not mean the user needs to be > named like this, but the configuration entry referencing any user should be > named like this. > Having a confiugration allows to have a different users per flow. This might > also be needed in case you have multiple Solr clusters. > Related discussion > [http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201912.mbox/browser] > SolrJ Kerberos integration: > [https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr] > Jaas conf documentation: > [https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/LoginConfigFile.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CONNECTORS-1629) Support Solr Kerberos Authentication
[
https://issues.apache.org/jira/browse/CONNECTORS-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007086#comment-17007086
]
Karl Wright commented on CONNECTORS-1629:
-
Hi,
I suggest we make changes piecemeal. First, updating the Jetty version, and
the jars that are included, as described here:
{quote}
You need jetty-client-9.4.25.v20191220.jar (maybe a slightly older 9.4.x
version will do as well, the current manifold version not). Reason is that you
will get otherwise a java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication error.
I was not exactly sure how to add this jar to the finally generated
distribution of ManifoldCF so i copied it in collector-lib and added it to the
classpath.
{quote}
To do this, we'd want to update the version of jetty specified in build.xml and
pom.xml, and add the new jar to the jetty jar list in build.xml. Then, in
framework/build.xml, the new jar should be added wherever jetty jars are found.
{quote}
I had to also deactivate the ModifiedLbSolrClient (commented out below)
otherwise you get an auth error 401. I believe the reason is that the default
SPNEGO Protocol for HTTP Kerberos always returns 401 not auth and THEN you are
supposed to do the Kerberos authentication, which is what SolrJ does
{quote}
The modified client is present because we need to be sure that the correct
(overridden) version of the SolrHttpClient class is used, not the default one.
So in this case you'd want to create a fresh copy of LBSolrClient and modify it
accordingly.
{quote}
Finally, you need to add to options.env.unix or options.env.win:
-Djava.security.auth.login.config=/path/to/jaas-client.conf
{quote}
I would suggest adding both the config file and the -D switch to all the
examples, but leave kerberos disabled unless somebody modifies the
jaas-client.conf file.
> Support Solr Kerberos Authentication
>
>
> Key: CONNECTORS-1629
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1629
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Solr 7.x component
>Affects Versions: ManifoldCF 2.14
>Reporter: Jörn Franke
>Priority: Major
>
> Several enterprise deployments of Solr are leveraging SolrCloud Kerberos
> authentication.
> The integration seems to be rather simple and the goal of this Jira is to
> evaluate the possential needed step to eventually contribute the Kerberos
> integration to the ManifoldCF project.
> The following steps would be needed:
> * One can pass the JVM parameter java.security.auth.login.config to the
> ManifoldCF JVM using -Djava.security.auth.login.config=/path/to/jaas.confg in
> which Kerberos authentication details, such as keytab and principal that has
> the right access to Solr is configured
> * A small adaption to the SolrCloudClient that is used within Manifold needs
> to be done to enable Kerberos authentication:
> HttpClientUtil.setConfigurer(new Krb5HttpClientConfigurer());
> Should this be integrated in Manifold, one may want to consider one input
> field in the configuration in the UI where one can select / flow which user
> defined in the Jaas conf (you can define multiple one) should be chosen. By
> default one may simply select "client" or "SolrJClient" if Jaas.conf is
> present in the System properties. This does not mean the user needs to be
> named like this, but the configuration entry referencing any user should be
> named like this.
> Having a confiugration allows to have a different users per flow. This might
> also be needed in case you have multiple Solr clusters.
> Related discussion
> [http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201912.mbox/browser]
> SolrJ Kerberos integration:
> [https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr]
> Jaas conf documentation:
> [https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/LoginConfigFile.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CONNECTORS-1629) Support Solr Kerberos Authentication
[
https://issues.apache.org/jira/browse/CONNECTORS-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007079#comment-17007079
]
Jörn Franke edited comment on CONNECTORS-1629 at 1/2/20 9:07 PM:
-
Hi,
I have some results. My test environment is RHEL, but it should work under
Windows with no issues (In fact i have a similar application Windows in Java
using a similar configuration to connet to a Kerberos Solr). I used Apache
Manifold 2.15 (JDK8 Coretto, you might need for Oracle JDK unlimited strength
policies) together with a Kerberos enabled Solr 8.3 with Zookeeper 3.5 as well
Kerberos enabled.
You need jetty-client-9.4.25.v20191220.jar (maybe a slightly older 9.4.x
version will do as well, the current manifold version not). Reason is that you
will get otherwise a java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication error.
I was not exactly sure how to add this jar to the finally generated
distribution of ManifoldCF so i copied it in collector-lib and added it to the
classpath.
Then, I did the following modifications (you will notice they slightly differ
from the one documented in the Solr reference guide):
[https://github.com/apache/manifoldcf/blob/trunk/connectors/solr/connector/src/main/java/org/apache/manifoldcf/agents/output/solr/HttpPoster.java]
[..]
{code:java}
Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder();
SolrHttpClientBuilder kb = krbBuild.getBuilder();
HttpClientUtil.setHttpClientBuilder(kb);
{code}
[..]
I had to also deactivate the ModifiedLbSolrClient (commented out below)
otherwise you get an auth error 401. I believe the reason is that the default
SPNEGO Protocol for HTTP Kerberos always returns 401 not auth and THEN you are
supposed to do the Kerberos authentication, which is what SolrJ does:
[..]
{code:java}
CloudSolrClient cloudSolrServer = new CloudSolrClient.Builder()
.withZkHost(zookeeperHosts)
//.withLBHttpSolrClient(new
ModifiedLBHttpSolrClient(HttpClientUtil.createClient(null), allowCompression))
.build();{code}
[..]
I dont know what the exact implications are, but for me it worked out fine.
CloudSolrClient does automatic loadbalancing based on the hosts found in
Zookeepr and thus the LBHttpSolrClient will not matter much, but I dont know
what the intention of the ModifiedLBHttpSolrClient was/is - let me know and i
will reinvestigate.
Then, you need to create a jaas-client.conf file, e.g. the following works:
{{jaas-client.conf:}}
{code:java}
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/path/to/keytab"
storeKey=true
useTicketCache=false
debug=true
principal="principal@DOMAIN";
};{code}
Finally, you need to add to options.env.unix or options.env.win:
-Djava.security.auth.login.config=/path/to/jaas-client.conf
Furthemore, I propose to have a configuration item or at least in the Manifold
documentation how to activate Kerberos.
I am happy to contribute this as well as a Github pull request or similar.
Please let me also know what you think about the approach. I think it is worth
to follow as many Enterprises use Kerberos.
best regards
was (Author: jornfranke):
Hi,
I have some results. My test environment is RHEL, but it should work under
Windows with no issues (In fact i have a similar application Windows in Java
using a similar configuration to connet to a Kerberos Solr). I used Apache
Manifold 2.15 (JDK8 Coretto, you might need for Oracle JDK unlimited strength
policies) together with a Kerberos enabled Solr 8.3 with Zookeeper 3.5 as well
Kerberos enabled.
You need jetty-client-9.4.25.v20191220.jar (maybe a slightly older 9.4.x
version will do as well, the current manifold version not). Reason is that you
will get otherwise a java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication error.
I was not exactly sure how to add this jar to the finally generated
distribution of ManifoldCF so i copied it in collector-lib and added it to the
classpath.
Then, I did the following modifications (you will notice they slightly differ
from the one documented in the Solr reference guide):
[https://github.com/apache/manifoldcf/blob/trunk/connectors/solr/connector/src/main/java/org/apache/manifoldcf/agents/output/solr/HttpPoster.java]
[..]
Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder();
SolrHttpClientBuilder kb = krbBuild.getBuilder();
HttpClientUtil.setHttpClientBuilder(kb);
[..]
I had to also deactivate the ModifiedLbSolrClient otherwise you get an auth
error. I believe the reason is that the default SPNEGO Protocol for HTTP
Kerberos always returns 401 not auth and THEN you are supposed to do the
Kerberos authentication, which is what SolrJ does:
[..]
{quote}CloudSolrClient cloudSolrServer = new CloudSolrClient.Builder()
.withZkHost(zookeeperHosts)
[jira] [Comment Edited] (CONNECTORS-1629) Support Solr Kerberos Authentication
[
https://issues.apache.org/jira/browse/CONNECTORS-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007079#comment-17007079
]
Jörn Franke edited comment on CONNECTORS-1629 at 1/2/20 9:06 PM:
-
Hi,
I have some results. My test environment is RHEL, but it should work under
Windows with no issues (In fact i have a similar application Windows in Java
using a similar configuration to connet to a Kerberos Solr). I used Apache
Manifold 2.15 (JDK8 Coretto, you might need for Oracle JDK unlimited strength
policies) together with a Kerberos enabled Solr 8.3 with Zookeeper 3.5 as well
Kerberos enabled.
You need jetty-client-9.4.25.v20191220.jar (maybe a slightly older 9.4.x
version will do as well, the current manifold version not). Reason is that you
will get otherwise a java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication error.
I was not exactly sure how to add this jar to the finally generated
distribution of ManifoldCF so i copied it in collector-lib and added it to the
classpath.
Then, I did the following modifications (you will notice they slightly differ
from the one documented in the Solr reference guide):
[https://github.com/apache/manifoldcf/blob/trunk/connectors/solr/connector/src/main/java/org/apache/manifoldcf/agents/output/solr/HttpPoster.java]
[..]
Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder();
SolrHttpClientBuilder kb = krbBuild.getBuilder();
HttpClientUtil.setHttpClientBuilder(kb);
[..]
I had to also deactivate the ModifiedLbSolrClient otherwise you get an auth
error. I believe the reason is that the default SPNEGO Protocol for HTTP
Kerberos always returns 401 not auth and THEN you are supposed to do the
Kerberos authentication, which is what SolrJ does:
[..]
{quote}CloudSolrClient cloudSolrServer = new CloudSolrClient.Builder()
.withZkHost(zookeeperHosts)
//.withLBHttpSolrClient(new
ModifiedLBHttpSolrClient(HttpClientUtil.createClient(null), allowCompression))
.build();
{quote}
[..]
I dont know what the exact implications are, but for me it worked out fine.
CloudSolrClient does automatic loadbalancing based on the hosts found in
Zookeepr and thus the LBHttpSolrClient will not matter much, but I dont know
what the intention of the ModifiedLBHttpSolrClient was/is - let me know and i
will reinvestigate.
Then, you need to create a jaas-client.conf file, e.g. the following works:
{{jaas-client.conf:}}
{code:java}
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/path/to/keytab"
storeKey=true
useTicketCache=false
debug=true
principal="principal@DOMAIN";
};{code}
Finally, you need to add to options.env.unix or options.env.win:
-Djava.security.auth.login.config=/path/to/jaas-client.conf
Furthemore, I propose to have a configuration item or at least in the Manifold
documentation how to activate Kerberos.
I am happy to contribute this as well as a Github pull request or similar.
Please let me also know what you think about the approach. I think it is worth
to follow as many Enterprises use Kerberos.
best regards
was (Author: jornfranke):
Hi,
I have some results. My test environment is RHEL, but it should work under
Windows with no issues (In fact i have a similar application Windows in Java
using a similar configuration to connet to a Kerberos Solr). I used Apache
Manifold 2.15 (JDK8 Coretto, you might need for Oracle JDK unlimited strength
policies) together with a Kerberos enabled Solr 8.3 with Zookeeper 3.5 as well
Kerberos enabled.
You need jetty-client-9.4.25.v20191220.jar (maybe a slightly older 9.4.x
version will do as well, the current manifold version not). Reason is that you
will get otherwise a java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication error.
I was not exactly sure how to add this jar to the finally generated
distribution of ManifoldCF so i copied it in collector-lib and added it to the
classpath.
Then, I did the following modifications (you will notice they slightly differ
from the one documented in the Solr reference guide):
[https://github.com/apache/manifoldcf/blob/trunk/connectors/solr/connector/src/main/java/org/apache/manifoldcf/agents/output/solr/HttpPoster.java]
[..]
Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder();
SolrHttpClientBuilder kb = krbBuild.getBuilder();
HttpClientUtil.setHttpClientBuilder(kb);
[..]
I had to also deactivate the ModifiedLbSolrClient otherwise you get an auth
error. I believe the reason is that the default SPNEGO Protocol for HTTP
Kerberos always returns 401 not auth and THEN you are supposed to do the
Kerberos authentication, which is what SolrJ does:
[..]
{quote}CloudSolrClient cloudSolrServer = new CloudSolrClient.Builder()
.withZkHost(zookeeperHosts)
//.withLBHttpSolrClient(new
[jira] [Comment Edited] (CONNECTORS-1629) Support Solr Kerberos Authentication
[
https://issues.apache.org/jira/browse/CONNECTORS-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007079#comment-17007079
]
Jörn Franke edited comment on CONNECTORS-1629 at 1/2/20 9:05 PM:
-
Hi,
I have some results. My test environment is RHEL, but it should work under
Windows with no issues (In fact i have a similar application Windows in Java
using a similar configuration to connet to a Kerberos Solr). I used Apache
Manifold 2.15 (JDK8 Coretto, you might need for Oracle JDK unlimited strength
policies) together with a Kerberos enabled Solr 8.3 with Zookeeper 3.5 as well
Kerberos enabled.
You need jetty-client-9.4.25.v20191220.jar (maybe a slightly older 9.4.x
version will do as well, the current manifold version not). Reason is that you
will get otherwise a java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication error.
I was not exactly sure how to add this jar to the finally generated
distribution of ManifoldCF so i copied it in collector-lib and added it to the
classpath.
Then, I did the following modifications (you will notice they slightly differ
from the one documented in the Solr reference guide):
[https://github.com/apache/manifoldcf/blob/trunk/connectors/solr/connector/src/main/java/org/apache/manifoldcf/agents/output/solr/HttpPoster.java]
[..]
Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder();
SolrHttpClientBuilder kb = krbBuild.getBuilder();
HttpClientUtil.setHttpClientBuilder(kb);
[..]
I had to also deactivate the ModifiedLbSolrClient otherwise you get an auth
error. I believe the reason is that the default SPNEGO Protocol for HTTP
Kerberos always returns 401 not auth and THEN you are supposed to do the
Kerberos authentication, which is what SolrJ does:
[..]
{quote}CloudSolrClient cloudSolrServer = new CloudSolrClient.Builder()
.withZkHost(zookeeperHosts)
//.withLBHttpSolrClient(new
ModifiedLBHttpSolrClient(HttpClientUtil.createClient(null), allowCompression))
.build();
{quote}
[..]
I dont know what the exact implications are, but for me it worked out fine.
CloudSolrClient does automatic loadbalancing based on the hosts found in
Zookeepr and thus the LBHttpSolrClient will not matter much, but I dont know
what the intention of the ModifiedLBHttpSolrClient was/is - let me know and i
will reinvestigate.
Then, you need to create a jaas-client.conf file, e.g. the following works:
{{jaas-client.conf:}}
{{C}}{{lient}}{{ { }}
{{com.sun.security.auth.module.Krb5LoginModule required }}
{{useKeyTab=true }}
{{keyTab="/path/to/keytab" }}
{{storeKey=true }}
{{useTicketCache=false }}
{{d}}{{ebug=true }}
{{principal="principal@DOMAIN"; }}}{{;}}
Finally, you need to add to options.env.unix or options.env.win:
-Djava.security.auth.login.config=/path/to/jaas-client.conf
Furthemore, I propose to have a configuration item or at least in the Manifold
documentation how to activate Kerberos.
I am happy to contribute this as well as a Github pull request or similar.
Please let me also know what you think about the approach. I think it is worth
to follow as many Enterprises use Kerberos.
best regards
was (Author: jornfranke):
Hi,
I have some results. My test environment is RHEL, but it should work under
Windows with no issues (In fact i have a similar application Windows in Java
using a similar configuration to connet to a Kerberos Solr). I used Apache
Manifold 2.15 (JDK8 Coretto, you might need for Oracle JDK unlimited strength
policies) together with a Kerberos enabled Solr 8.3 with Zookeeper 3.5 as well
Kerberos enabled.
You need jetty-client-9.4.25.v20191220.jar (maybe a slightly older 9.4.x
version will do as well, the current manifold version not). Reason is that you
will get otherwise a java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication error.
I was not exactly sure how to add this jar to the finally generated
distribution of ManifoldCF so i copied it in collector-lib and added it to the
classpath.
Then, I did the following modifications (you will notice they slightly differ
from the one documented in the Solr reference guide):
[https://github.com/apache/manifoldcf/blob/trunk/connectors/solr/connector/src/main/java/org/apache/manifoldcf/agents/output/solr/HttpPoster.java]
[..]
Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder();
SolrHttpClientBuilder kb = krbBuild.getBuilder();
HttpClientUtil.setHttpClientBuilder(kb);
[..]
I had to also deactivate the ModifiedLbSolrClient otherwise you get an auth
error. I believe the reason is that the default SPNEGO Protocol for HTTP
Kerberos always returns 401 not auth and THEN you are supposed to do the
Kerberos authentication, which is what SolrJ does:
[..]
{quote}CloudSolrClient cloudSolrServer = new CloudSolrClient.Builder()
.withZkHost(zookeeperHosts)
[jira] [Comment Edited] (CONNECTORS-1629) Support Solr Kerberos Authentication
[
https://issues.apache.org/jira/browse/CONNECTORS-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007079#comment-17007079
]
Jörn Franke edited comment on CONNECTORS-1629 at 1/2/20 9:04 PM:
-
Hi,
I have some results. My test environment is RHEL, but it should work under
Windows with no issues (In fact i have a similar application Windows in Java
using a similar configuration to connet to a Kerberos Solr). I used Apache
Manifold 2.15 (JDK8 Coretto, you might need for Oracle JDK unlimited strength
policies) together with a Kerberos enabled Solr 8.3 with Zookeeper 3.5 as well
Kerberos enabled.
You need jetty-client-9.4.25.v20191220.jar (maybe a slightly older 9.4.x
version will do as well, the current manifold version not). Reason is that you
will get otherwise a java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication error.
I was not exactly sure how to add this jar to the finally generated
distribution of ManifoldCF so i copied it in collector-lib and added it to the
classpath.
Then, I did the following modifications (you will notice they slightly differ
from the one documented in the Solr reference guide):
[https://github.com/apache/manifoldcf/blob/trunk/connectors/solr/connector/src/main/java/org/apache/manifoldcf/agents/output/solr/HttpPoster.java]
[..]
Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder();
SolrHttpClientBuilder kb = krbBuild.getBuilder();
HttpClientUtil.setHttpClientBuilder(kb);
[..]
I had to also deactivate the ModifiedLbSolrClient otherwise you get an auth
error. I believe the reason is that the default SPNEGO Protocol for HTTP
Kerberos always returns 401 not auth and THEN you are supposed to do the
Kerberos authentication, which is what SolrJ does:
[..]
{quote}CloudSolrClient cloudSolrServer = new CloudSolrClient.Builder()
.withZkHost(zookeeperHosts)
//.withLBHttpSolrClient(new
ModifiedLBHttpSolrClient(HttpClientUtil.createClient(null), allowCompression))
.build();
{quote}
[..]
I dont know what the exact implications are, but for me it worked out fine.
CloudSolrClient does automatic loadbalancing based on the hosts found in
Zookeepr and thus the LBHttpSolrClient will not matter much, but I dont know
what the intention of the ModifiedLBHttpSolrClient was/is - let me know and i
will reinvestigate.
Then, you need to create a jaas-client.conf file, e.g. the following works:
jaas-client.conf
{quote}
Client { com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/path/to/keytab"
storeKey=true
useTicketCache=false
debug=true
principal="principal@DOMAIN"; };
{quote}
Finally, you need to add to options.env.unix or options.env.win:
-Djava.security.auth.login.config=/path/to/jaas-client.conf
Furthemore, I propose to have a configuration item or at least in the Manifold
documentation how to activate Kerberos.
I am happy to contribute this as well as a Github pull request or similar.
Please let me also know what you think about the approach. I think it is worth
to follow as many Enterprises use Kerberos.
best regards
was (Author: jornfranke):
Hi,
I have some results. My test environment is RHEL, but it should work under
Windows with no issues (In fact i have a similar application Windows in Java
using a similar configuration to connet to a Kerberos Solr). I used Apache
Manifold 2.15 (JDK8 Coretto, you might need for Oracle JDK unlimited strength
policies) together with a Kerberos enabled Solr 8.3 with Zookeeper 3.5 as well
Kerberos enabled.
You need jetty-client-9.4.25.v20191220.jar (maybe a slightly older 9.4.x
version will do as well, the current manifold version not). Reason is that you
will get otherwise a java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication error.
I was not exactly sure how to add this jar to the finally generated
distribution of ManifoldCF so i copied it in collector-lib and added it to the
classpath.
Then, I did the following modifications (you will notice they slightly differ
from the one documented in the Solr reference guide):
[https://github.com/apache/manifoldcf/blob/trunk/connectors/solr/connector/src/main/java/org/apache/manifoldcf/agents/output/solr/HttpPoster.java]
[..]
Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder();
SolrHttpClientBuilder kb = krbBuild.getBuilder();
HttpClientUtil.setHttpClientBuilder(kb);
[..]
I had to also deactivate the ModifiedLbSolrClient otherwise you get an auth
error. I believe the reason is that the default SPNEGO Protocol for HTTP
Kerberos always returns 401 not auth and THEN you are supposed to do the
Kerberos authentication, which is what SolrJ does:
[..]
{quote}CloudSolrClient cloudSolrServer = new CloudSolrClient.Builder()
.withZkHost(zookeeperHosts)
//.withLBHttpSolrClient(new
[jira] [Comment Edited] (CONNECTORS-1629) Support Solr Kerberos Authentication
[
https://issues.apache.org/jira/browse/CONNECTORS-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007079#comment-17007079
]
Jörn Franke edited comment on CONNECTORS-1629 at 1/2/20 9:03 PM:
-
Hi,
I have some results. My test environment is RHEL, but it should work under
Windows with no issues (In fact i have a similar application Windows in Java
using a similar configuration to connet to a Kerberos Solr). I used Apache
Manifold 2.15 (JDK8 Coretto, you might need for Oracle JDK unlimited strength
policies) together with a Kerberos enabled Solr 8.3 with Zookeeper 3.5 as well
Kerberos enabled.
You need jetty-client-9.4.25.v20191220.jar (maybe a slightly older 9.4.x
version will do as well, the current manifold version not). Reason is that you
will get otherwise a java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication error.
I was not exactly sure how to add this jar to the finally generated
distribution of ManifoldCF so i copied it in collector-lib and added it to the
classpath.
Then, I did the following modifications (you will notice they slightly differ
from the one documented in the Solr reference guide):
[https://github.com/apache/manifoldcf/blob/trunk/connectors/solr/connector/src/main/java/org/apache/manifoldcf/agents/output/solr/HttpPoster.java]
[..]
Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder();
SolrHttpClientBuilder kb = krbBuild.getBuilder();
HttpClientUtil.setHttpClientBuilder(kb);
[..]
I had to also deactivate the ModifiedLbSolrClient otherwise you get an auth
error. I believe the reason is that the default SPNEGO Protocol for HTTP
Kerberos always returns 401 not auth and THEN you are supposed to do the
Kerberos authentication, which is what SolrJ does:
[..]
{quote}CloudSolrClient cloudSolrServer = new CloudSolrClient.Builder()
.withZkHost(zookeeperHosts)
//.withLBHttpSolrClient(new
ModifiedLBHttpSolrClient(HttpClientUtil.createClient(null), allowCompression))
.build();
{quote}
[..]
I dont know what the exact implications are, but for me it worked out fine.
CloudSolrClient does automatic loadbalancing based on the hosts found in
Zookeepr and thus the LBHttpSolrClient will not matter much, but I dont know
what the intention of the ModifiedLBHttpSolrClient was/is - let me know and i
will reinvestigate.
Then, you need to create a jaas-client.conf file, e.g. the following works:
{quote}jaas-client.conf:
Client
{ com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/path/to/keytab"
storeKey=true
useTicketCache=false
debug=true
principal="principal@DOMAIN"; };
{quote}
Finally, you need to add to options.env.unix or options.env.win:
-Djava.security.auth.login.config=/path/to/jaas-client.conf
Furthemore, I propose to have a configuration item or at least in the Manifold
documentation how to activate Kerberos.
I am happy to contribute this as well as a Github pull request or similar.
Please let me also know what you think about the approach. I think it is worth
to follow as many Enterprises use Kerberos.
best regards
was (Author: jornfranke):
Hi,
I have some results. My test environment is RHEL, but it should work under
Windows with no issues (In fact i have a similar application Windows in Java
using a similar configuration to connet to a Kerberos Solr). I used Apache
Manifold 2.15 (JDK8 Coretto, you might need for Oracle JDK unlimited strength
policies) together with a Kerberos enabled Solr 8.3 with Zookeeper 3.5 as well
Kerberos enabled.
You need jetty-client-9.4.25.v20191220.jar (maybe a slightly older 9.4.x
version will do as well, the current manifold version not). Reason is that you
will get otherwise a java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication error.
I was not exactly sure how to add this jar to the finally generated
distribution of ManifoldCF so i copied it in collector-lib and added it to the
classpath.
Then, I did the following modifications (you will notice they slightly differ
from the one documented in the Solr reference guide):
[https://github.com/apache/manifoldcf/blob/trunk/connectors/solr/connector/src/main/java/org/apache/manifoldcf/agents/output/solr/HttpPoster.java]
[..]
Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder();
SolrHttpClientBuilder kb = krbBuild.getBuilder();
HttpClientUtil.setHttpClientBuilder(kb);
[..]
I had to also deactivate the ModifiedLbSolrClient otherwise you get an auth
error. I believe the reason is that the default SPNEGO Protocol for HTTP
Kerberos always returns 401 not auth and THEN you are supposed to do the
Kerberos authentication, which is what SolrJ does:
[..]
CloudSolrClient cloudSolrServer = new CloudSolrClient.Builder()
.withZkHost(zookeeperHosts)
//.withLBHttpSolrClient(new
[jira] [Commented] (CONNECTORS-1629) Support Solr Kerberos Authentication
[
https://issues.apache.org/jira/browse/CONNECTORS-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007079#comment-17007079
]
Jörn Franke commented on CONNECTORS-1629:
-
Hi,
I have some results. My test environment is RHEL, but it should work under
Windows with no issues (In fact i have a similar application Windows in Java
using a similar configuration to connet to a Kerberos Solr). I used Apache
Manifold 2.15 (JDK8 Coretto, you might need for Oracle JDK unlimited strength
policies) together with a Kerberos enabled Solr 8.3 with Zookeeper 3.5 as well
Kerberos enabled.
You need jetty-client-9.4.25.v20191220.jar (maybe a slightly older 9.4.x
version will do as well, the current manifold version not). Reason is that you
will get otherwise a java.lang.ClassNotFoundException:
org.eclipse.jetty.client.util.SPNEGOAuthentication error.
I was not exactly sure how to add this jar to the finally generated
distribution of ManifoldCF so i copied it in collector-lib and added it to the
classpath.
Then, I did the following modifications (you will notice they slightly differ
from the one documented in the Solr reference guide):
[https://github.com/apache/manifoldcf/blob/trunk/connectors/solr/connector/src/main/java/org/apache/manifoldcf/agents/output/solr/HttpPoster.java]
[..]
Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder();
SolrHttpClientBuilder kb = krbBuild.getBuilder();
HttpClientUtil.setHttpClientBuilder(kb);
[..]
I had to also deactivate the ModifiedLbSolrClient otherwise you get an auth
error. I believe the reason is that the default SPNEGO Protocol for HTTP
Kerberos always returns 401 not auth and THEN you are supposed to do the
Kerberos authentication, which is what SolrJ does:
[..]
CloudSolrClient cloudSolrServer = new CloudSolrClient.Builder()
.withZkHost(zookeeperHosts)
//.withLBHttpSolrClient(new
ModifiedLBHttpSolrClient(HttpClientUtil.createClient(null), allowCompression))
.build();
[..]
I dont know what the exact implications are, but for me it worked out fine.
CloudSolrClient does automatic loadbalancing based on the hosts found in
Zookeepr and thus the LBHttpSolrClient will not matter much, but I dont know
what the intention of the ModifiedLBHttpSolrClient was/is - let me know and i
will reinvestigate.
Then, you need to create a jaas-client.conf file, e.g. the following works:
jaas-client.conf:
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/path/to/keytab"
storeKey=true
useTicketCache=false
debug=true
principal="principal@DOMAIN";
};
Finally, you need to add to options.env.unix or options.env.win:
-Djava.security.auth.login.config=/path/to/jaas-client.conf
Furthemore, I propose to have a configuration item or at least in the Manifold
documentation how to activate Kerberos.
I am happy to contribute this as well as a Github pull request or similar.
Please let me also know what you think about the approach. I think it is worth
to follow as many Enterprises use Kerberos.
best regards
> Support Solr Kerberos Authentication
>
>
> Key: CONNECTORS-1629
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1629
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Solr 7.x component
>Affects Versions: ManifoldCF 2.14
>Reporter: Jörn Franke
>Priority: Major
>
> Several enterprise deployments of Solr are leveraging SolrCloud Kerberos
> authentication.
> The integration seems to be rather simple and the goal of this Jira is to
> evaluate the possential needed step to eventually contribute the Kerberos
> integration to the ManifoldCF project.
> The following steps would be needed:
> * One can pass the JVM parameter java.security.auth.login.config to the
> ManifoldCF JVM using -Djava.security.auth.login.config=/path/to/jaas.confg in
> which Kerberos authentication details, such as keytab and principal that has
> the right access to Solr is configured
> * A small adaption to the SolrCloudClient that is used within Manifold needs
> to be done to enable Kerberos authentication:
> HttpClientUtil.setConfigurer(new Krb5HttpClientConfigurer());
> Should this be integrated in Manifold, one may want to consider one input
> field in the configuration in the UI where one can select / flow which user
> defined in the Jaas conf (you can define multiple one) should be chosen. By
> default one may simply select "client" or "SolrJClient" if Jaas.conf is
> present in the System properties. This does not mean the user needs to be
> named like this, but the configuration entry referencing any user should be
> named like this.
> Having a confiugration allows to have a different users per flow. This might
> also be needed in case you have multiple Solr clusters.
> Related
