Lune
The version before Ranger 0.6 might not work well with authentication. Even though, it might have been easy for us to support basic or digest auth, but I think, we missed it. The background is, Solr 5.2 introduced support for Kerberos and Solr 5.3 started natively supporting Basic Auth. However, Solr 5.2 also upgraded their HTTP Client libraries which were much newer than the jars used by Hadoop. For that reason, Ranger couldn’t use the new native authentication from Solr. In Ranger 0.6, we now have isolation for jars used by Ranger plugin. This enabled us to address conflicting jars. The Ranger 0.6 supports Kerberos out of the box. The best option for you is to update the Ranger 0.5 code base to read user/password from the plugin configuration file and use them in the SolrAuditDestination java class. And replace the plugin jars for the component you are using. Thanks Bosco From: Lune Silver <lunescar.ran...@gmail.com> Reply-To: <user@ranger.incubator.apache.org> Date: Friday, August 26, 2016 at 8:57 AM To: <user@ranger.incubator.apache.org> Subject: Audit to secure solr with digest authentication Hello ! I'm trying to use SolR as a storage for ranger audit, but I'm encountering one blocking problem. I'm using HDP 2.3.4.7 and Ambari 2.2.2. In Ambari, for audit on solR, I have two fields - ranger.audit.solr.username - ranger.audit.solr.password I log in the ranger admin UI and check the audit part and it just says there is no audit. When I check the logs from ranger-admin (in DEBUG mode), I can see a 401 error. ### 2016-08-26 17:22:28,874 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.DefaultClientConnectionOperator (DefaultClientConnectionOperator.java:177) - Connecting to <SOLR HOST>:6083 2016-08-26 17:22:28,902 [http-bio-6182-exec-10] DEBUG org.apache.http.client.protocol.RequestAddCookies (RequestAddCookies.java:132) - CookieSpec selected: best-match 2016-08-26 17:22:28,902 [http-bio-6182-exec-4] DEBUG org.apache.http.client.protocol.RequestAddCookies (RequestAddCookies.java:132) - CookieSpec selected: best-match 2016-08-26 17:22:28,915 [http-bio-6182-exec-10] DEBUG org.apache.http.client.protocol.RequestAuthCache (RequestAuthCache.java:78) - Auth cache not set in the context 2016-08-26 17:22:28,915 [http-bio-6182-exec-4] DEBUG org.apache.http.client.protocol.RequestAuthCache (RequestAuthCache.java:78) - Auth cache not set in the context 2016-08-26 17:22:28,915 [http-bio-6182-exec-10] DEBUG org.apache.http.client.protocol.RequestTargetAuthentication (RequestTargetAuthentication.java:78) - Target auth state: UNCHALLENGED 2016-08-26 17:22:28,915 [http-bio-6182-exec-4] DEBUG org.apache.http.client.protocol.RequestTargetAuthentication (RequestTargetAuthentication.java:78) - Target auth state: UNCHALLENGED 2016-08-26 17:22:28,916 [http-bio-6182-exec-10] DEBUG org.apache.http.client.protocol.RequestProxyAuthentication (RequestProxyAuthentication.java:87) - Proxy auth state: UNCHALLENGED 2016-08-26 17:22:28,916 [http-bio-6182-exec-4] DEBUG org.apache.http.client.protocol.RequestProxyAuthentication (RequestProxyAuthentication.java:87) - Proxy auth state: UNCHALLENGED 2016-08-26 17:22:28,916 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.client.DefaultRequestDirector (DefaultRequestDirector.java:713) - Attempt 1 to execute request 2016-08-26 17:22:28,916 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.client.DefaultRequestDirector (DefaultRequestDirector.java:713) - Attempt 1 to execute request 2016-08-26 17:22:28,917 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:269) - Sending request: GET /solr/ranger_audits/select?q=*%3A*&fq=evtTime%3A%5B2016-08-25T22%3A00%3A00Z+TO+NOW%5D&sort=evtTime+desc&start=0&rows=25&wt=javabin&version=2 HTTP/1.1 2016-08-26 17:22:28,917 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:269) - Sending request: GET /solr/ranger_audits/select?q=*%3A*&fq=evtTime%3A%5B2016-08-25T22%3A00%3A00Z+TO+NOW%5D&sort=evtTime+desc&start=0&rows=25&wt=javabin&version=2 HTTP/1.1 2016-08-26 17:22:28,917 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - >> "GET /solr/ranger_audits/select?q=*%3A*&fq=evtTime%3A%5B2016-08-25T22%3A00%3A00Z+TO+NOW%5D&sort=evtTime+desc&start=0&rows=25&wt=javabin&version=2 HTTP/1.1[\r][\n]" 2016-08-26 17:22:28,917 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - >> "GET /solr/ranger_audits/select?q=*%3A*&fq=evtTime%3A%5B2016-08-25T22%3A00%3A00Z+TO+NOW%5D&sort=evtTime+desc&start=0&rows=25&wt=javabin&version=2 HTTP/1.1[\r][\n]" 2016-08-26 17:22:28,918 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - >> "User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrClient] 1.0[\r][\n]" 2016-08-26 17:22:28,918 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - >> "User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrClient] 1.0[\r][\n]" 2016-08-26 17:22:28,919 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - >> "Host: <SOLR HOST>:6083[\r][\n]" 2016-08-26 17:22:28,919 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - >> "Host: <SOLR HOST>:6083[\r][\n]" 2016-08-26 17:22:28,919 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - >> "Connection: Keep-Alive[\r][\n]" 2016-08-26 17:22:28,919 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - >> "Connection: Keep-Alive[\r][\n]" 2016-08-26 17:22:28,920 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - >> "Accept-Encoding: gzip, deflate[\r][\n]" 2016-08-26 17:22:28,920 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - >> "Accept-Encoding: gzip, deflate[\r][\n]" 2016-08-26 17:22:28,920 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - >> "[\r][\n]" 2016-08-26 17:22:28,920 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - >> "[\r][\n]" 2016-08-26 17:22:28,921 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:273) - >> GET /solr/ranger_audits/select?q=*%3A*&fq=evtTime%3A%5B2016-08-25T22%3A00%3A00Z+TO+NOW%5D&sort=evtTime+desc&start=0&rows=25&wt=javabin&version=2 HTTP/1.1 2016-08-26 17:22:28,921 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:273) - >> GET /solr/ranger_audits/select?q=*%3A*&fq=evtTime%3A%5B2016-08-25T22%3A00%3A00Z+TO+NOW%5D&sort=evtTime+desc&start=0&rows=25&wt=javabin&version=2 HTTP/1.1 2016-08-26 17:22:28,921 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:276) - >> User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrClient] 1.0 2016-08-26 17:22:28,921 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:276) - >> User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrClient] 1.0 2016-08-26 17:22:28,922 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:276) - >> Host: <SOLR HOST>:6083 2016-08-26 17:22:28,922 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:276) - >> Connection: Keep-Alive 2016-08-26 17:22:28,922 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:276) - >> Accept-Encoding: gzip, deflate 2016-08-26 17:22:28,921 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:276) - >> Host: <SOLR HOST>:6083 2016-08-26 17:22:28,923 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:276) - >> Connection: Keep-Alive 2016-08-26 17:22:28,923 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:276) - >> Accept-Encoding: gzip, deflate 2016-08-26 17:22:28,923 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - << "HTTP/1.1 401 Unauthorized request, Response code: 401[\r][\n]" 2016-08-26 17:22:28,925 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - << "WWW-Authenticate: Basic realm="solr"[\r][\n]" 2016-08-26 17:22:28,925 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - << "Content-Type: text/html;charset=iso-8859-1[\r][\n]" 2016-08-26 17:22:28,925 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - << "Cache-Control: must-revalidate,no-cache,no-store[\r][\n]" 2016-08-26 17:22:28,926 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - << "Content-Length: 319[\r][\n]" 2016-08-26 17:22:28,926 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - << "[\r][\n]" 2016-08-26 17:22:28,927 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:254) - Receiving response: HTTP/1.1 401 Unauthorized request, Response code: 401 2016-08-26 17:22:28,927 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:257) - << HTTP/1.1 401 Unauthorized request, Response code: 401 2016-08-26 17:22:28,927 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:260) - << WWW-Authenticate: Basic realm="solr" 2016-08-26 17:22:28,927 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:260) - << Content-Type: text/html;charset=iso-8859-1 2016-08-26 17:22:28,927 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:260) - << Cache-Control: must-revalidate,no-cache,no-store 2016-08-26 17:22:28,928 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.conn.DefaultClientConnection (DefaultClientConnection.java:260) - << Content-Length: 319 2016-08-26 17:22:28,930 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.client.DefaultRequestDirector (DefaultRequestDirector.java:543) - Connection can be kept alive indefinitely 2016-08-26 17:22:28,930 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.client.HttpAuthenticator (HttpAuthenticator.java:70) - Authentication required 2016-08-26 17:22:28,930 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.client.HttpAuthenticator (HttpAuthenticator.java:97) - <SOLR HOST>:6083 requested authentication 2016-08-26 17:22:28,931 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.client.AuthenticationStrategyImpl (AuthenticationStrategyImpl.java:173) - Authentication schemes in the order of preference: [negotiate, Kerberos, NTLM, Digest, Basic] 2016-08-26 17:22:28,931 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.client.AuthenticationStrategyImpl (AuthenticationStrategyImpl.java:201) - Challenge for negotiate authentication scheme not available 2016-08-26 17:22:28,931 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.client.AuthenticationStrategyImpl (AuthenticationStrategyImpl.java:201) - Challenge for Kerberos authentication scheme not available 2016-08-26 17:22:28,931 [http-bio-6182-exec-10] DEBUG org.apache.http.impl.client.AuthenticationStrategyImpl (AuthenticationStrategyImpl.java:201) - Challenge for NTLM authentication scheme not available 2016-08-26 17:22:28,932 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - << "HTTP/1.1 401 Unauthorized request, Response code: 401[\r][\n]" 2016-08-26 17:22:28,932 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - << "WWW-Authenticate: Basic realm="solr"[\r][\n]" 2016-08-26 17:22:28,932 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - << "Content-Type: text/html;charset=iso-8859-1[\r][\n]" 2016-08-26 17:22:28,933 [http-bio-6182-exec-4] DEBUG org.apache.http.impl.conn.Wire (Wire.java:63) - << "Cache-Control: must-revalidate,no-cache,no-store[\r][\n]" ### When I put directly inside the REST API URL for SolR the login and password, it works fine. But with these properties, I have the 401 error. When I check the github,I see no mention of any username or password for solr audit in the class "SolrAuditDestination". https://github.com/hortonworks/ranger-release/blob/HDP-2.3.4.7-tag/agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java And it is the same for the HDP 2.4 : https://github.com/hortonworks/ranger-release/blob/HDP-2.4.2.18-tag/agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java Is it normal ? Is there a way for me to use SolR with Digest authentication in my version of Ranger in HDP 2.3.4.7 ? Thank you in advance ! Best regards. Lune.
