[jira] [Updated] (HADOOP-13368) DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation
[
https://issues.apache.org/jira/browse/HADOOP-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HADOOP-13368:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks [~jnp] for the review and commit to {{trunk}}, {{branch-2}} and
{{branch-2.8}}.
> DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should
> be O(1) operation
> -
>
> Key: HADOOP-13368
> URL: https://issues.apache.org/jira/browse/HADOOP-13368
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HADOOP-13368.000.patch, HADOOP-13368.001.patch
>
>
> To lookup, {{DFSOpsCountStatistics$OpType#fromSymbol}} and
> {{s3a.Statistic#fromSymbol}} iterates all the enums to get the entry by its
> symbol. Usages of {{fromSymbol()}} include {{isTracked()}} and {{getLong()}}.
> As there are dozens of enum entries, it merits to make these two similar
> operations O(1) complexity. This point is especially true if downstream app
> probes a dozen of stats in an outer loop (see [TEZ-3331]).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13368) DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation
[
https://issues.apache.org/jira/browse/HADOOP-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HADOOP-13368:
---
Attachment: HADOOP-13368.001.patch
Per offline discussion with [~Hitesh], the v1 patch setsthe initial capacity of
the hashmap to account for the load factor as we know the size of the enum.
> DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should
> be O(1) operation
> -
>
> Key: HADOOP-13368
> URL: https://issues.apache.org/jira/browse/HADOOP-13368
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HADOOP-13368.000.patch, HADOOP-13368.001.patch
>
>
> To lookup, {{DFSOpsCountStatistics$OpType#fromSymbol}} and
> {{s3a.Statistic#fromSymbol}} iterates all the enums to get the entry by its
> symbol. Usages of {{fromSymbol()}} include {{isTracked()}} and {{getLong()}}.
> As there are dozens of enum entries, it merits to make these two similar
> operations O(1) complexity. This point is especially true if downstream app
> probes a dozen of stats in an outer loop (see [TEZ-3331]).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13368) DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation
[
https://issues.apache.org/jira/browse/HADOOP-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HADOOP-13368:
---
Status: Patch Available (was: Open)
> DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should
> be O(1) operation
> -
>
> Key: HADOOP-13368
> URL: https://issues.apache.org/jira/browse/HADOOP-13368
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HADOOP-13368.000.patch
>
>
> To lookup, {{DFSOpsCountStatistics$OpType#fromSymbol}} and
> {{s3a.Statistic#fromSymbol}} iterates all the enums to get the entry by its
> symbol. Usages of {{fromSymbol()}} include {{isTracked()}} and {{getLong()}}.
> As there are dozens of enum entries, it merits to make these two similar
> operations O(1) complexity. This point is especially true if downstream app
> probes a dozen of stats in an outer loop (see [TEZ-3331]).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13368) DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation
[
https://issues.apache.org/jira/browse/HADOOP-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HADOOP-13368:
---
Attachment: HADOOP-13368.000.patch
The v0 patch uses a pre-built private static final hashmap to speed up the
{{fromSymbol()}} lookup.
> DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should
> be O(1) operation
> -
>
> Key: HADOOP-13368
> URL: https://issues.apache.org/jira/browse/HADOOP-13368
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs
>Affects Versions: 2.8.0
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HADOOP-13368.000.patch
>
>
> To lookup, {{DFSOpsCountStatistics$OpType#fromSymbol}} and
> {{s3a.Statistic#fromSymbol}} iterates all the enums to get the entry by its
> symbol. Usages of {{fromSymbol()}} include {{isTracked()}} and {{getLong()}}.
> As there are dozens of enum entries, it merits to make these two similar
> operations O(1) complexity. This point is especially true if downstream app
> probes a dozen of stats in an outer loop (see [TEZ-3331]).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
