[jira] [Updated] (HADOOP-13442) Optimize UGI group lookups
[ https://issues.apache.org/jira/browse/HADOOP-13442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-13442: - Attachment: HADOOP-13442.patch # Changed group provider to cache the de-dupped list instead of the raw list. # Added new {{UGI#getGroups}} that returns the aforementioned de-duped list # Changed {{UGI#getPrimaryGroup}} to call {{UGI#getGroups}} to avoid an array copy # Removed unnecessary synchronization of {{UGI#getGroups}} method. Required minor tweak to {{Groups#getGroups}} to be thread-safe. Already used elsewhere w/o synch, so this just makes it safe. Reduces contention with cached token->ugi instances. > Optimize UGI group lookups > -- > > Key: HADOOP-13442 > URL: https://issues.apache.org/jira/browse/HADOOP-13442 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Attachments: HADOOP-13442.patch > > > {{UGI#getGroups}} and its usage is inefficient. The list is unnecessarily > converted to multiple collections. > For _every_ invocation, the {{List}} from the group provider is > converted into a {{LinkedHashSet}} (to de-dup), back to a > {{String[]}}. Then callers testing for group membership convert back to a > {{List}}. This should be done once to reduce allocations. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13442) Optimize UGI group lookups
[ https://issues.apache.org/jira/browse/HADOOP-13442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-13442: - Status: Patch Available (was: Open) Will be much more useful with hdfs counterpart. > Optimize UGI group lookups > -- > > Key: HADOOP-13442 > URL: https://issues.apache.org/jira/browse/HADOOP-13442 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Attachments: HADOOP-13442.patch > > > {{UGI#getGroups}} and its usage is inefficient. The list is unnecessarily > converted to multiple collections. > For _every_ invocation, the {{List}} from the group provider is > converted into a {{LinkedHashSet}} (to de-dup), back to a > {{String[]}}. Then callers testing for group membership convert back to a > {{List}}. This should be done once to reduce allocations. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13442) Optimize UGI group lookups
[ https://issues.apache.org/jira/browse/HADOOP-13442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HADOOP-13442: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 2.8.0 Status: Resolved (was: Patch Available) Committed to trunk, branch-2 and branch-2.8. > Optimize UGI group lookups > -- > > Key: HADOOP-13442 > URL: https://issues.apache.org/jira/browse/HADOOP-13442 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HADOOP-13442.patch > > > {{UGI#getGroups}} and its usage is inefficient. The list is unnecessarily > converted to multiple collections. > For _every_ invocation, the {{List}} from the group provider is > converted into a {{LinkedHashSet}} (to de-dup), back to a > {{String[]}}. Then callers testing for group membership convert back to a > {{List}}. This should be done once to reduce allocations. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13442) Optimize UGI group lookups
[ https://issues.apache.org/jira/browse/HADOOP-13442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HADOOP-13442: - Fix Version/s: 2.7.5 Just committed along with broken tests HDFS-10738 and MAPREDUCE-6750 to branch-2.7. Thank you Daryn. Updating Fix Version for all three. > Optimize UGI group lookups > -- > > Key: HADOOP-13442 > URL: https://issues.apache.org/jira/browse/HADOOP-13442 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Fix For: 2.8.0, 3.0.0-alpha1, 2.7.5 > > Attachments: HADOOP-13442.patch > > > {{UGI#getGroups}} and its usage is inefficient. The list is unnecessarily > converted to multiple collections. > For _every_ invocation, the {{List}} from the group provider is > converted into a {{LinkedHashSet}} (to de-dup), back to a > {{String[]}}. Then callers testing for group membership convert back to a > {{List}}. This should be done once to reduce allocations. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org