[jira] [Commented] (RANGER-4481) Add a configuration item to support Ranger client not using authentication
[
https://issues.apache.org/jira/browse/RANGER-4481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17783990#comment-17783990
]
Xuze Yang commented on RANGER-4481:
---
This problem can be reproduced everytime according to the following steps:
Environment: 2 ranger servers（called ranger0 and ranger1）, 1 kerberos
1. Restart the ranger client (to ensure that the ranger client has
interacted with only one ranger server)
2. Determine the range server that the range client reqeust to, assuming
it is range0
3. Make the kerberos unavailable
4. Make ranger0 unavailable
5. Make the kerberos available
A little explanation: In step 4, by making range0 unavailable, the ranger
client will switch to interacting with range1. After the first HTTP request
returns 401, the ranger client will request a ticket for ranger1 from kerberos.
However, due to kerberos's unavailable at this time, the ticket request failed.
And Because it's the first time for ranger client interact with ranger1,
NegotiateAuthentication#supported will put the entry: \{ranger1's hostname,
false}, and it will continue to return false in subsequent requests.
After applying the provided patch, the ranger client can pull policies after
kerberos returns to normal.
> Add a configuration item to support Ranger client not using authentication
> --
>
> Key: RANGER-4481
> URL: https://issues.apache.org/jira/browse/RANGER-4481
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Xuze Yang
>Assignee: Xuze Yang
>Priority: Major
> Attachments: feasible solution code.png, first http response.png,
> openjdk problem code.png, second http request.png
>
>
> As described in RANGER-3602, ranger supports downloading policies and roles
> through unauthenticated http requests even if kerberos is enabled on the
> server.
> But in terms of the current implementation of RangerAdminRESTClient, whether
> to enable authenticated HTTP requests depends on the service in which it is
> located. For example, if the Hadoop service has kerberos enabled, then the
> RangerAdminRESTClient in the HDFS and Yarn plugins will also use
> authenticated HTTP requests.
> I think this is not reasonable enough. In this case (both the Ranger server
> and Hadoop are enabled for kerberos), the RangerAdminRESTClient of the HDFS
> and Yarn plugins should also be allowed to download policies and roles
> through unauthenticated HTTP requests.
> The reason why I proposed this improvement is due to a bug I encountered in
> our production environment. I will introduce the bug I encountered later.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (RANGER-4481) Add a configuration item to support Ranger client not using authentication
[ https://issues.apache.org/jira/browse/RANGER-4481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776988#comment-17776988 ] Xuze Yang commented on RANGER-4481: --- Add a configuration item to enable RangerAdminRESTClient's getRolesIfUpdated()/getServicePoliciesIfUpdated()/getServiceTagsIfUpdated() use unauthenticated http request may involve a large amount of work. Because we should add this configuration item in all plugin component's configuration file. Another way, when the response code was 401, I tried to clear the supported cache through java reflection. This has been proven to be feasible. !4.png! Now I don't know which modification method is more reasonable, or there are other better modification methods. [~madhan] [~kirbyzhou] > Add a configuration item to support Ranger client not using authentication > -- > > Key: RANGER-4481 > URL: https://issues.apache.org/jira/browse/RANGER-4481 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 2.1.0 >Reporter: Xuze Yang >Priority: Major > Attachments: 1.png, 2.png, 3.png, 4.png > > > As described in RANGER-3602, ranger supports downloading policies and roles > through unauthenticated http requests even if kerberos is enabled on the > server. > But in terms of the current implementation of RangerAdminRESTClient, whether > to enable authenticated HTTP requests depends on the service in which it is > located. For example, if the Hadoop service has kerberos enabled, then the > RangerAdminRESTClient in the HDFS and Yarn plugins will also use > authenticated HTTP requests. > I think this is not reasonable enough. In this case (both the Ranger server > and Hadoop are enabled for kerberos), the RangerAdminRESTClient of the HDFS > and Yarn plugins should also be allowed to download policies and roles > through unauthenticated HTTP requests. > The reason why I proposed this improvement is due to a bug I encountered in > our production environment. I will introduce the bug I encountered later. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4481) Add a configuration item to support Ranger client not using authentication
[ https://issues.apache.org/jira/browse/RANGER-4481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776964#comment-17776964 ] Xuze Yang commented on RANGER-4481: --- This is a bug in the openjdk code. Someone in the openjdk community has already raised an issue（[https://bugs.openjdk.org/browse/JDK-8208299]）, but the issue is still in an open state, and this issue still exists in the latest version of openjdk. Therefore, it is necessary for us to provide a method to avoid the problem on the ranger side. CC [~madhan] [~kirbyzhou] > Add a configuration item to support Ranger client not using authentication > -- > > Key: RANGER-4481 > URL: https://issues.apache.org/jira/browse/RANGER-4481 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 2.1.0 >Reporter: Xuze Yang >Priority: Major > Attachments: 1.png, 2.png, 3.png > > > As described in RANGER-3602, ranger supports downloading policies and roles > through unauthenticated http requests even if kerberos is enabled on the > server. > But in terms of the current implementation of RangerAdminRESTClient, whether > to enable authenticated HTTP requests depends on the service in which it is > located. For example, if the Hadoop service has kerberos enabled, then the > RangerAdminRESTClient in the HDFS and Yarn plugins will also use > authenticated HTTP requests. > I think this is not reasonable enough. In this case (both the Ranger server > and Hadoop are enabled for kerberos), the RangerAdminRESTClient of the HDFS > and Yarn plugins should also be allowed to download policies and roles > through unauthenticated HTTP requests. > The reason why I proposed this improvement is due to a bug I encountered in > our production environment. I will introduce the bug I encountered later. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4481) Add a configuration item to support Ranger client not using authentication
[
https://issues.apache.org/jira/browse/RANGER-4481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776958#comment-17776958
]
Xuze Yang commented on RANGER-4481:
---
{*}Problem phenomenon{*}:
>From a certain moment on, the ranger plugin of resourcemanager0 has been
>unable to pull policies, and the error log is 401. The kerberos service is
>normal, and as a comparison, the ranger plugin of resourcemanager1 can pull
>policies normally. Resourcemanager0's error log:
{code:java}
2023-10-19 08:55:02,551 WARN
org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting Roles.
secureMode=true,
user=hadoop/hdp-hadoop-hdp-resourcemanager-1.hdp-hadoop-hdp-resourcemanager.yangxuze.svc.cluster.lo...@dahua.com
(auth:KERBEROS), response={"httpStatusCode":401,"statusCode":0},
serviceName=default-Yarn
2023-10-19 08:55:02,552 WARN
org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies.
secureMode=true,
user=hadoop/hdp-hadoop-hdp-resourcemanager-1.hdp-hadoop-hdp-resourcemanager.yangxuze.svc.cluster.lo...@dahua.com
(auth:KERBEROS), response={"httpStatusCode":401,"statusCode":0},
serviceName=default-Yarn
2023-10-19 08:55:12,572 WARN
org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting Roles.
secureMode=true,
user=hadoop/hdp-hadoop-hdp-resourcemanager-1.hdp-hadoop-hdp-resourcemanager.yangxuze.svc.cluster.lo...@dahua.com
(auth:KERBEROS), response={"httpStatusCode":401,"statusCode":0},
serviceName=default-Yarn
2023-10-19 08:55:12,574 WARN
org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies.
secureMode=true,
user=hadoop/hdp-hadoop-hdp-resourcemanager-1.hdp-hadoop-hdp-resourcemanager.yangxuze.svc.cluster.lo...@dahua.com
(auth:KERBEROS), response={"httpStatusCode":401,"statusCode":0},
serviceName=default-Yarn{code}
*Problem Cause Analysis：*
The normal process for the Ranger plugin to pull policies from the Ranger
server：
1) For the first request, the server will reply with a 401 and carry "WWW
Authenticate: Negotiate" in the HTTP header
!1.png!
2) The internal httpUrlConnection class of Openjdk handles this by performing a
kerberos authentication, and then placing the authentication result in the HTTP
header field Authorization to request again
!2.png!
3) Afterwards, the server can provide the correct response, such as a 200
response or a 304 response
When performing kerberos authentication, if the kdc service is abnormal, it
will be determined that the service does not support kerberos authentication,
and then this information will be stored in a global static map called
'supported'. Subsequent requests will be obtained from the 'supported' based on
the server hostname to determine whether authentication methods are supported
!3.png!
*Therefore, once the first failure occurs, it will be determined that the
service does not support authentication, and the second HTTP request will never
be triggered, resulting in the inability to obtain policies from the service
forever.*
> Add a configuration item to support Ranger client not using authentication
> --
>
> Key: RANGER-4481
> URL: https://issues.apache.org/jira/browse/RANGER-4481
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Xuze Yang
>Priority: Major
> Attachments: 1.png, 2.png, 3.png
>
>
> As described in RANGER-3602, ranger supports downloading policies and roles
> through unauthenticated http requests even if kerberos is enabled on the
> server.
> But in terms of the current implementation of RangerAdminRESTClient, whether
> to enable authenticated HTTP requests depends on the service in which it is
> located. For example, if the Hadoop service has kerberos enabled, then the
> RangerAdminRESTClient in the HDFS and Yarn plugins will also use
> authenticated HTTP requests.
> I think this is not reasonable enough. In this case (both the Ranger server
> and Hadoop are enabled for kerberos), the RangerAdminRESTClient of the HDFS
> and Yarn plugins should also be allowed to download policies and roles
> through unauthenticated HTTP requests.
> The reason why I proposed this improvement is due to a bug I encountered in
> our production environment. I will introduce the bug I encountered later.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
