[ https://issues.apache.org/jira/browse/BEAM-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308245#comment-15308245 ]
Frances Perry commented on BEAM-315: ------------------------------------ The Beam Model (and hence the DataflowRunner) specifies that GoupByKey compares for key equality using encoded elements. https://github.com/apache/incubator-beam/blob/638cc20719cb314d9e6b29fa79d45edcf4586dce/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupByKey.java#L50 That's also why GroupByKey.validate() ensures the deterministic coder requirements -- if the user does a GBK, they must provide a key coder that is known to produce the same encoding for equal keys. https://github.com/apache/incubator-beam/blob/638cc20719cb314d9e6b29fa79d45edcf4586dce/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupByKey.java#L188 > GroupByKey/CoGroupByKey doesn't group correctly in batch mode of > FlinkPipelineRunner > ------------------------------------------------------------------------------------ > > Key: BEAM-315 > URL: https://issues.apache.org/jira/browse/BEAM-315 > Project: Beam > Issue Type: Bug > Components: runner-flink > Affects Versions: 0.1.0-incubating > Reporter: Pawel Szczur > Attachments: CoGroupPipelineStringKey.java > > > Same keys are processed multiple times. > A repo to reproduce the bug: > https://github.com/orian/cogroup-wrong-grouping > Discussion: > http://mail-archives.apache.org/mod_mbox/incubator-beam-user/201605.mbox/%3CCAB2uKkG2xHsWpLFUkYnt8eEzdxU%3DB_nu6crTwVi-ZuUpugxkPQ%40mail.gmail.com%3E > Notice: I haven't tested other runners (didn't manage to configure Spark). -- This message was sent by Atlassian JIRA (v6.3.4#6332)