[
https://issues.apache.org/jira/browse/CRUNCH-479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14186178#comment-14186178
]
Micah Whitacre commented on CRUNCH-479:
---------------------------------------
Change looks good, but -1 to some of the subtle assertions in the test code.
> Writing to target with WriteMode.APPEND merges values into PCollection
> ----------------------------------------------------------------------
>
> Key: CRUNCH-479
> URL: https://issues.apache.org/jira/browse/CRUNCH-479
> Project: Crunch
> Issue Type: Bug
> Components: Core
> Reporter: Micah Whitacre
> Assignee: Josh Wills
> Attachments: CRUNCH-479.patch, CRUNCH-479b.patch
>
>
> This was mentioned as part of CDK-617[1]. A PCollection that contains a set
> of values, is written to a target with WriteMode.APPEND, and then that
> PCollection is materialized, when you iterate over that PCollection it
> contains not only the new values that were appended but also the existing
> values. This is surprising as most would expect that collection to only
> contain the original collection of values. A use case for this might be if
> the solution is looking to only process the new values instead of dealing
> with all of the existing data.
> [1] - https://issues.cloudera.org/browse/CDK-671
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
