[jira] [Commented] (RANGER-2894) Plugins cannot interact with Solr with basic auth as audit targer
[
https://issues.apache.org/jira/browse/RANGER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706210#comment-17706210
]
Jiayi Liu commented on RANGER-2894:
---
[~rmani] I submitted a patch, can you take a look?
> Plugins cannot interact with Solr with basic auth as audit targer
> -
>
> Key: RANGER-2894
> URL: https://issues.apache.org/jira/browse/RANGER-2894
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Nikita Ilyushkin
>Priority: Major
> Attachments: 001-RANGER-2894.patch
>
>
> There seems to be a problem with audit to Solr with [basic
> authentication|https://lucene.apache.org/solr/guide/8_1/basic-authentication-plugin.html].
> With the simple Solr cloud setup with basic auth every plugin I tried (HDFS,
> YARN, HBase, Hive) failed to write audit to it with the similar errors:
> {code:java}
> 2020-06-25T19:39:35,248 ERROR
> [hiveServer2.async.batch_hiveServer2.async.batch.solr_destWriter]
> impl.CloudSolrClient: Request to collection [ranger_audits] failed due to
> (401) org.apache.solr.client.solrj.impl.H
> ttpSolrClient$RemoteSolrException: Error from server at
> http://nilyushkin-hadoop-dev-0.ru-central1.internal:8983/solr/ranger_audits_shard1_replica_n1:
> Expected mime type application/octet-stream but got text/htm
> l.
>
>
> Error 401 require authentication
>
> HTTP ERROR 401
> Problem accessing /solr/ranger_audits_shard1_replica_n1/update. Reason:
> require authentication
>
>
> {code}
> tcpdump confirms that no auth headers are in requests.
> Content of ranger--audit.xml:
> {code:java}
>
> xasecure.audit.is.enabled
> true
>
>
> xasecure.audit.destination.solr
> true
>
>
> xasecure.audit.destination.solr.urls
> http://fqdn:8983/solr/ranger_audits
>
>
> xasecure.audit.destination.solr.user
> rangeraudit
>
>
> xasecure.audit.destination.solr.password
> admin
>
>
> xasecure.audit.destination.solr.zookeepers
> fqdn:2181/solr.server
>
>
> xasecure.audit.destination.solr.batch.filespool.dir
> /srv/audit_solr_spool
>
> {code}
> The same results with xasecure.audit.destination.solr.urls instead
> xasecure.audit.destination.solr.zookeepers.
> Ranger Admin on the other hand writes audit just fine with given credentials
> to the same Solr.
> Unsurprisingly, following Solr documentation (underlying solrj really) and
> adding:
> {code:java}
> -Dsolr.httpclient.builder.factory=org.apache.solr.client.solrj.impl.PreemptiveBasicAuthClientBuilderFactory
> -Dbasicauth=rangeraudit:admin
> {code}
> to the audited daemon (like HiveServer2) solves the problem.
> I also haven't found setBasicAuthCredentials (the second method of auth
> solrj provides) in the plugin sources or packages, so I assume it's just not
> implemented or bugged.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-2894) Plugins cannot interact with Solr with basic auth as audit targer
[
https://issues.apache.org/jira/browse/RANGER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17705984#comment-17705984
]
Jiayi Liu commented on RANGER-2894:
---
[~rmani] Hi, I solved this problem by using System.setProperty to set
solr.httpclient.builder.factory and basicauth in SolrAuditDestination. Do you
think it's ok to do this? I can submit a PR, thanks.
> Plugins cannot interact with Solr with basic auth as audit targer
> -
>
> Key: RANGER-2894
> URL: https://issues.apache.org/jira/browse/RANGER-2894
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Nikita Ilyushkin
>Priority: Major
>
> There seems to be a problem with audit to Solr with [basic
> authentication|https://lucene.apache.org/solr/guide/8_1/basic-authentication-plugin.html].
> With the simple Solr cloud setup with basic auth every plugin I tried (HDFS,
> YARN, HBase, Hive) failed to write audit to it with the similar errors:
> {code:java}
> 2020-06-25T19:39:35,248 ERROR
> [hiveServer2.async.batch_hiveServer2.async.batch.solr_destWriter]
> impl.CloudSolrClient: Request to collection [ranger_audits] failed due to
> (401) org.apache.solr.client.solrj.impl.H
> ttpSolrClient$RemoteSolrException: Error from server at
> http://nilyushkin-hadoop-dev-0.ru-central1.internal:8983/solr/ranger_audits_shard1_replica_n1:
> Expected mime type application/octet-stream but got text/htm
> l.
>
>
> Error 401 require authentication
>
> HTTP ERROR 401
> Problem accessing /solr/ranger_audits_shard1_replica_n1/update. Reason:
> require authentication
>
>
> {code}
> tcpdump confirms that no auth headers are in requests.
> Content of ranger--audit.xml:
> {code:java}
>
> xasecure.audit.is.enabled
> true
>
>
> xasecure.audit.destination.solr
> true
>
>
> xasecure.audit.destination.solr.urls
> http://fqdn:8983/solr/ranger_audits
>
>
> xasecure.audit.destination.solr.user
> rangeraudit
>
>
> xasecure.audit.destination.solr.password
> admin
>
>
> xasecure.audit.destination.solr.zookeepers
> fqdn:2181/solr.server
>
>
> xasecure.audit.destination.solr.batch.filespool.dir
> /srv/audit_solr_spool
>
> {code}
> The same results with xasecure.audit.destination.solr.urls instead
> xasecure.audit.destination.solr.zookeepers.
> Ranger Admin on the other hand writes audit just fine with given credentials
> to the same Solr.
> Unsurprisingly, following Solr documentation (underlying solrj really) and
> adding:
> {code:java}
> -Dsolr.httpclient.builder.factory=org.apache.solr.client.solrj.impl.PreemptiveBasicAuthClientBuilderFactory
> -Dbasicauth=rangeraudit:admin
> {code}
> to the audited daemon (like HiveServer2) solves the problem.
> I also haven't found setBasicAuthCredentials (the second method of auth
> solrj provides) in the plugin sources or packages, so I assume it's just not
> implemented or bugged.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-2894) Plugins cannot interact with Solr with basic auth as audit targer
[
https://issues.apache.org/jira/browse/RANGER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195175#comment-17195175
]
Ramesh Mani commented on RANGER-2894:
-
[~mallniya] Basic auth is no supported for now. If you working on a basic auth
client, please provide a patch for the same. Also recommend way to to use SSL
also along with basic auth because of security concerns, for this its better to
use kerberos.
Following doc have kerberos configuration detailed.
https://community.cloudera.com/t5/Community-Articles/Securing-Solr-Collections-with-Ranger-Kerberos/ta-p/245519
> Plugins cannot interact with Solr with basic auth as audit targer
> -
>
> Key: RANGER-2894
> URL: https://issues.apache.org/jira/browse/RANGER-2894
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Nikita Ilyushkin
>Priority: Major
>
> There seems to be a problem with audit to Solr with [basic
> authentication|https://lucene.apache.org/solr/guide/8_1/basic-authentication-plugin.html].
> With the simple Solr cloud setup with basic auth every plugin I tried (HDFS,
> YARN, HBase, Hive) failed to write audit to it with the similar errors:
> {code:java}
> 2020-06-25T19:39:35,248 ERROR
> [hiveServer2.async.batch_hiveServer2.async.batch.solr_destWriter]
> impl.CloudSolrClient: Request to collection [ranger_audits] failed due to
> (401) org.apache.solr.client.solrj.impl.H
> ttpSolrClient$RemoteSolrException: Error from server at
> http://nilyushkin-hadoop-dev-0.ru-central1.internal:8983/solr/ranger_audits_shard1_replica_n1:
> Expected mime type application/octet-stream but got text/htm
> l.
>
>
> Error 401 require authentication
>
> HTTP ERROR 401
> Problem accessing /solr/ranger_audits_shard1_replica_n1/update. Reason:
> require authentication
>
>
> {code}
> tcpdump confirms that no auth headers are in requests.
> Content of ranger--audit.xml:
> {code:java}
>
> xasecure.audit.is.enabled
> true
>
>
> xasecure.audit.destination.solr
> true
>
>
> xasecure.audit.destination.solr.urls
> http://fqdn:8983/solr/ranger_audits
>
>
> xasecure.audit.destination.solr.user
> rangeraudit
>
>
> xasecure.audit.destination.solr.password
> admin
>
>
> xasecure.audit.destination.solr.zookeepers
> fqdn:2181/solr.server
>
>
> xasecure.audit.destination.solr.batch.filespool.dir
> /srv/audit_solr_spool
>
> {code}
> The same results with xasecure.audit.destination.solr.urls instead
> xasecure.audit.destination.solr.zookeepers.
> Ranger Admin on the other hand writes audit just fine with given credentials
> to the same Solr.
> Unsurprisingly, following Solr documentation (underlying solrj really) and
> adding:
> {code:java}
> -Dsolr.httpclient.builder.factory=org.apache.solr.client.solrj.impl.PreemptiveBasicAuthClientBuilderFactory
> -Dbasicauth=rangeraudit:admin
> {code}
> to the audited daemon (like HiveServer2) solves the problem.
> I also haven't found setBasicAuthCredentials (the second method of auth
> solrj provides) in the plugin sources or packages, so I assume it's just not
> implemented or bugged.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2894) Plugins cannot interact with Solr with basic auth as audit targer
[
https://issues.apache.org/jira/browse/RANGER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195137#comment-17195137
]
Nikita Ilyushkin commented on RANGER-2894:
--
No. Neither I wanted it in this installation nor I found requirement of
Kerberos in documentation.
> Plugins cannot interact with Solr with basic auth as audit targer
> -
>
> Key: RANGER-2894
> URL: https://issues.apache.org/jira/browse/RANGER-2894
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Nikita Ilyushkin
>Priority: Major
>
> There seems to be a problem with audit to Solr with [basic
> authentication|https://lucene.apache.org/solr/guide/8_1/basic-authentication-plugin.html].
> With the simple Solr cloud setup with basic auth every plugin I tried (HDFS,
> YARN, HBase, Hive) failed to write audit to it with the similar errors:
> {code:java}
> 2020-06-25T19:39:35,248 ERROR
> [hiveServer2.async.batch_hiveServer2.async.batch.solr_destWriter]
> impl.CloudSolrClient: Request to collection [ranger_audits] failed due to
> (401) org.apache.solr.client.solrj.impl.H
> ttpSolrClient$RemoteSolrException: Error from server at
> http://nilyushkin-hadoop-dev-0.ru-central1.internal:8983/solr/ranger_audits_shard1_replica_n1:
> Expected mime type application/octet-stream but got text/htm
> l.
>
>
> Error 401 require authentication
>
> HTTP ERROR 401
> Problem accessing /solr/ranger_audits_shard1_replica_n1/update. Reason:
> require authentication
>
>
> {code}
> tcpdump confirms that no auth headers are in requests.
> Content of ranger--audit.xml:
> {code:java}
>
> xasecure.audit.is.enabled
> true
>
>
> xasecure.audit.destination.solr
> true
>
>
> xasecure.audit.destination.solr.urls
> http://fqdn:8983/solr/ranger_audits
>
>
> xasecure.audit.destination.solr.user
> rangeraudit
>
>
> xasecure.audit.destination.solr.password
> admin
>
>
> xasecure.audit.destination.solr.zookeepers
> fqdn:2181/solr.server
>
>
> xasecure.audit.destination.solr.batch.filespool.dir
> /srv/audit_solr_spool
>
> {code}
> The same results with xasecure.audit.destination.solr.urls instead
> xasecure.audit.destination.solr.zookeepers.
> Ranger Admin on the other hand writes audit just fine with given credentials
> to the same Solr.
> Unsurprisingly, following Solr documentation (underlying solrj really) and
> adding:
> {code:java}
> -Dsolr.httpclient.builder.factory=org.apache.solr.client.solrj.impl.PreemptiveBasicAuthClientBuilderFactory
> -Dbasicauth=rangeraudit:admin
> {code}
> to the audited daemon (like HiveServer2) solves the problem.
> I also haven't found setBasicAuthCredentials (the second method of auth
> solrj provides) in the plugin sources or packages, so I assume it's just not
> implemented or bugged.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2894) Plugins cannot interact with Solr with basic auth as audit targer
[
https://issues.apache.org/jira/browse/RANGER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194301#comment-17194301
]
Velmurugan Periasamy commented on RANGER-2894:
--
Have you tried with kerberos? CC [~rmani]
> Plugins cannot interact with Solr with basic auth as audit targer
> -
>
> Key: RANGER-2894
> URL: https://issues.apache.org/jira/browse/RANGER-2894
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 1.1.0, 2.0.0
>Reporter: Nikita Ilyushkin
>Priority: Major
>
> There seems to be a problem with audit to Solr with [basic
> authentication|https://lucene.apache.org/solr/guide/8_1/basic-authentication-plugin.html].
> With the simple Solr cloud setup with basic auth every plugin I tried (HDFS,
> YARN, HBase, Hive) failed to write audit to it with the similar errors:
> {code:java}
> 2020-06-25T19:39:35,248 ERROR
> [hiveServer2.async.batch_hiveServer2.async.batch.solr_destWriter]
> impl.CloudSolrClient: Request to collection [ranger_audits] failed due to
> (401) org.apache.solr.client.solrj.impl.H
> ttpSolrClient$RemoteSolrException: Error from server at
> http://nilyushkin-hadoop-dev-0.ru-central1.internal:8983/solr/ranger_audits_shard1_replica_n1:
> Expected mime type application/octet-stream but got text/htm
> l.
>
>
> Error 401 require authentication
>
> HTTP ERROR 401
> Problem accessing /solr/ranger_audits_shard1_replica_n1/update. Reason:
> require authentication
>
>
> {code}
> tcpdump confirms that no auth headers are in requests.
> Content of ranger--audit.xml:
> {code:java}
>
> xasecure.audit.is.enabled
> true
>
>
> xasecure.audit.destination.solr
> true
>
>
> xasecure.audit.destination.solr.urls
> http://fqdn:8983/solr/ranger_audits
>
>
> xasecure.audit.destination.solr.user
> rangeraudit
>
>
> xasecure.audit.destination.solr.password
> admin
>
>
> xasecure.audit.destination.solr.zookeepers
> fqdn:2181/solr.server
>
>
> xasecure.audit.destination.solr.batch.filespool.dir
> /srv/audit_solr_spool
>
> {code}
> The same results with xasecure.audit.destination.solr.urls instead
> xasecure.audit.destination.solr.zookeepers.
> Ranger Admin on the other hand writes audit just fine with given credentials
> to the same Solr.
> Unsurprisingly, following Solr documentation (underlying solrj really) and
> adding:
> {code:java}
> -Dsolr.httpclient.builder.factory=org.apache.solr.client.solrj.impl.PreemptiveBasicAuthClientBuilderFactory
> -Dbasicauth=rangeraudit:admin
> {code}
> to the audited daemon (like HiveServer2) solves the problem.
> I also haven't found setBasicAuthCredentials (the second method of auth
> solrj provides) in the plugin sources or packages, so I assume it's just not
> implemented or bugged.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
