[
https://issues.apache.org/jira/browse/FLINK-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144481#comment-15144481
]
ASF GitHub Bot commented on FLINK-3340:
---------------------------------------
Github user ggevay commented on the pull request:
https://github.com/apache/flink/pull/1626#issuecomment-183287868
So, if I understand correctly, this is aiming to fix the problem that was
brought up by [1], but in a different way then proposed there.
I still have the same worry about this solution that I wrote in this
comment: [2]: `MutableObjectIterator.next(reuse)` is used at 271 places,
according to IntelliJ. If we accept the change in the object reuse contract of
`MergeIterator.next(reuse)` introduced by
0a8df6d513fa59d650ff875bdf3a1613d0f14af5, all the call sites can potentially
have the same problem as the reduce drivers, so we should check all of these
271 places. For example, just by skimming the list for a few minutes, all the 8
calls in `CrossDriver` seem suspicious at first glance, as well as the calls in
`UnionWithTempOperator`, `MutableHashTable.ProbeIterator.next`, and
`ReusingBuildFirstHashJoinIterator.callWithNextKey`.
My alternative solution [3] has the drawback that it would introduce an
extra copying in `MergeIterator.HeadStream.getHead`, but would change the
contract of `MergeIterator.next(reuse)` back to the same as it was before
0a8df6d513fa59d650ff875bdf3a1613d0f14af5.
[1] https://issues.apache.org/jira/browse/FLINK-3291
[2]
https://issues.apache.org/jira/browse/FLINK-3291?focusedCommentId=15130452&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15130452
[3]
https://github.com/ggevay/flink/commit/b05e12075bb5d0518b6fe12cd8f7de1e6b16f805
> Fix object juggling in drivers
> ------------------------------
>
> Key: FLINK-3340
> URL: https://issues.apache.org/jira/browse/FLINK-3340
> Project: Flink
> Issue Type: Bug
> Components: Distributed Runtime
> Affects Versions: 1.0.0
> Reporter: Greg Hogan
> Assignee: Greg Hogan
> Priority: Critical
>
> {{ReduceDriver}}, {{ReduceCombineDriver}}, and {{ChainedAllReduceDriver}} are
> not properly tracking objects for reuse.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
