[
https://issues.apache.org/jira/browse/FLINK-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15563930#comment-15563930
]
ASF GitHub Bot commented on FLINK-4705:
---------------------------------------
GitHub user greghogan opened a pull request:
https://github.com/apache/flink/pull/2617
[FLINK-4705] Instrument FixedLengthRecordSorter
Updates comparators with support for key normalization.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/greghogan/flink
4705_instrument_fixedlengthrecordsorter
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2617.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2617
----
commit 10d0955251ad9965da2094682ea1d4c468602373
Author: Greg Hogan <[email protected]>
Date: 2016-09-28T16:55:03Z
[FLINK-4705] Instrument FixedLengthRecordSorter
Updates comparators with support for key normalization.
----
> Instrument FixedLengthRecordSorter
> ----------------------------------
>
> Key: FLINK-4705
> URL: https://issues.apache.org/jira/browse/FLINK-4705
> Project: Flink
> Issue Type: Improvement
> Components: Local Runtime
> Affects Versions: 1.2.0
> Reporter: Greg Hogan
> Assignee: Greg Hogan
>
> The {{NormalizedKeySorter}} sorts on the concatenation of (potentially
> partial) keys plus an 8-byte pointer to the record. After sorting each
> pointer must be dereferenced, which is not cache friendly.
> The {{FixedLengthRecordSorter}} sorts on the concatentation of full keys
> followed by the remainder of the record. The records can then be deserialized
> in sequence.
> Instrumenting the {{FixedLengthRecordSorter}} requires implementing the
> comparator methods {{writereadWithKeyNormalization}} and
> {{readWithKeyNormalization}}.
> Testing {{JaccardIndex}} on an m4.16xlarge the scale 18 runtime dropped from
> 71.8 to 68.8 s (4.3% faster) and the scale 20 runtime dropped from 546.1 to
> 501.8 s (8.8% faster).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
