[jira] [Updated] (HADOOP-13442) Optimize UGI group lookups
[
https://issues.apache.org/jira/browse/HADOOP-13442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HADOOP-13442:
-
Attachment: HADOOP-13442.patch
# Changed group provider to cache the de-dupped list instead of the raw list.
# Added new {{UGI#getGroups}} that returns the aforementioned de-duped list
# Changed {{UGI#getPrimaryGroup}} to call {{UGI#getGroups}} to avoid an array
copy
# Removed unnecessary synchronization of {{UGI#getGroups}} method. Required
minor tweak to {{Groups#getGroups}} to be thread-safe. Already used elsewhere
w/o synch, so this just makes it safe. Reduces contention with cached
token->ugi instances.
> Optimize UGI group lookups
> --
>
> Key: HADOOP-13442
> URL: https://issues.apache.org/jira/browse/HADOOP-13442
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
> Attachments: HADOOP-13442.patch
>
>
> {{UGI#getGroups}} and its usage is inefficient. The list is unnecessarily
> converted to multiple collections.
> For _every_ invocation, the {{List}} from the group provider is
> converted into a {{LinkedHashSet}} (to de-dup), back to a
> {{String[]}}. Then callers testing for group membership convert back to a
> {{List}}. This should be done once to reduce allocations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13442) Optimize UGI group lookups
[
https://issues.apache.org/jira/browse/HADOOP-13442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HADOOP-13442:
-
Status: Patch Available (was: Open)
Will be much more useful with hdfs counterpart.
> Optimize UGI group lookups
> --
>
> Key: HADOOP-13442
> URL: https://issues.apache.org/jira/browse/HADOOP-13442
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
> Attachments: HADOOP-13442.patch
>
>
> {{UGI#getGroups}} and its usage is inefficient. The list is unnecessarily
> converted to multiple collections.
> For _every_ invocation, the {{List}} from the group provider is
> converted into a {{LinkedHashSet}} (to de-dup), back to a
> {{String[]}}. Then callers testing for group membership convert back to a
> {{List}}. This should be done once to reduce allocations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13442) Optimize UGI group lookups
[
https://issues.apache.org/jira/browse/HADOOP-13442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HADOOP-13442:
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.0.0-alpha2
2.8.0
Status: Resolved (was: Patch Available)
Committed to trunk, branch-2 and branch-2.8.
> Optimize UGI group lookups
> --
>
> Key: HADOOP-13442
> URL: https://issues.apache.org/jira/browse/HADOOP-13442
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
> Fix For: 2.8.0, 3.0.0-alpha2
>
> Attachments: HADOOP-13442.patch
>
>
> {{UGI#getGroups}} and its usage is inefficient. The list is unnecessarily
> converted to multiple collections.
> For _every_ invocation, the {{List}} from the group provider is
> converted into a {{LinkedHashSet}} (to de-dup), back to a
> {{String[]}}. Then callers testing for group membership convert back to a
> {{List}}. This should be done once to reduce allocations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13442) Optimize UGI group lookups
[
https://issues.apache.org/jira/browse/HADOOP-13442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Shvachko updated HADOOP-13442:
-
Fix Version/s: 2.7.5
Just committed along with broken tests HDFS-10738 and MAPREDUCE-6750 to
branch-2.7. Thank you Daryn.
Updating Fix Version for all three.
> Optimize UGI group lookups
> --
>
> Key: HADOOP-13442
> URL: https://issues.apache.org/jira/browse/HADOOP-13442
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
> Fix For: 2.8.0, 3.0.0-alpha1, 2.7.5
>
> Attachments: HADOOP-13442.patch
>
>
> {{UGI#getGroups}} and its usage is inefficient. The list is unnecessarily
> converted to multiple collections.
> For _every_ invocation, the {{List}} from the group provider is
> converted into a {{LinkedHashSet}} (to de-dup), back to a
> {{String[]}}. Then callers testing for group membership convert back to a
> {{List}}. This should be done once to reduce allocations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
