[ https://issues.apache.org/jira/browse/BEAM-6294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743484#comment-16743484 ]
Thomas Weise commented on BEAM-6294: ------------------------------------ [~robertwb] [~mxm] we found that the non-native reshuffle expansion seems to have an issue. We saw records being dropped with Flink portable streaming. The issue went away with the switch to Flink rebalance. > Use Flink's redistribute for reshuffle. > --------------------------------------- > > Key: BEAM-6294 > URL: https://issues.apache.org/jira/browse/BEAM-6294 > Project: Beam > Issue Type: New Feature > Components: runner-flink, sdk-py-core > Reporter: Robert Bradshaw > Assignee: Robert Bradshaw > Priority: Major > Fix For: 2.10.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Python needs to publish the URN over the FnAPI which is pretty easy, but > Flink also needs to ensure that the composite structure does not get fused. > Unlike with GBK, we can't assume all runners implement this as a primitive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)