[
https://issues.apache.org/jira/browse/HIVE-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15702942#comment-15702942
]
Illya Yalovyy commented on HIVE-15076:
--------------------------------------
[~ngangam]
Thank you for the feedback on this CR.
Here is some performance results (what I can share):
|| User member of # groups || GroupMembershipKeyFilter ||
UserMembershipKeyFilter ||
| 200 | 0.118 | 0.103 |
| 400 | 0.135 | 0.106 |
| 600 | 0.171 | 0.113 |
| 800 | 0.230 | 0.119 |
| 1000 | FAIL | 0.129 |
GroupMembershipKeyFilter fails with "javax.naming.SizeLimitExceededException:
[LDAP: error code 4 - Sizelimit Exceeded]" when number of groups greater than
800. The particular number of groups when the default implementation fails
depends on record size for each group, so in real production it will be much
less.
> Improve scalability of LDAP authentication provider group filter
> ----------------------------------------------------------------
>
> Key: HIVE-15076
> URL: https://issues.apache.org/jira/browse/HIVE-15076
> Project: Hive
> Issue Type: Improvement
> Components: Authentication
> Affects Versions: 2.1.0
> Reporter: Illya Yalovyy
> Assignee: Illya Yalovyy
> Attachments: HIVE-15076.1.patch, HIVE-15076.2.patch
>
>
> Current implementation uses following algorithm:
> # For a given user find all groups that user is a member of. (A list of
> LDAP groups is constructed as a result of that request)
> # Match this list of groups with provided group filter.
>
> Time/Memory complexity of this approach is O(N) on client side, where N – is
> a number of groups the user has membership in. On a large directory (800+
> groups per user) we can observe up to 2x performance degradation and failures
> because of size of LDAP response (LDAP: error code 4 - Sizelimit Exceeded).
>
> Some Directory Services (Microsoft Active Directory for instance) provide a
> virtual attribute for User Object that contains a list of groups that user
> belongs to. This attribute can be used to quickly determine whether this user
> passes or fails the group filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
