[
https://issues.apache.org/jira/browse/FLINK-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14966670#comment-14966670
]
Stephan Ewen commented on FLINK-2882:
-------------------------------------
Interesting find!
I am curious, how often to we need to call this method, actually? When we
transfer IDs, we do not convert them to strings. So is this all done for the
sake of logging?
If so, is the logging by itself also becoming a bottleneck when increasing the
DOP?
> Improve performance of string conversions
> -----------------------------------------
>
> Key: FLINK-2882
> URL: https://issues.apache.org/jira/browse/FLINK-2882
> Project: Flink
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.10
> Reporter: Greg Hogan
> Assignee: Greg Hogan
>
> {{AbstractID.toString()}} and {{AbstractID.toShortString()}} call
> {{StringUtils.byteToHexString(...)}} which uses a StringBuilder to convert
> from binary to hex. This is a hotspot when scaling the number of workers.
> While testing on my single node with parallelism=512 jvisualvm reports
> 600,000 calls taking 13.4 seconds. Improving
> {{StringUtils.byteToHexString(...)}} reduces the time to 1.3 seconds.
> Additionally memoizing the string values in {{AbstractID}} reduce the time to
> 350 ms and the number of calls to {{StringUtils.byteToHexString(...)}} to
> ~1000.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
