[ https://issues.apache.org/jira/browse/PIG-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xuefu Zhang resolved PIG-4876. ------------------------------ Resolution: Fixed Committed to Spark branch. Thanks, Xianda. > OutputConsumeIterator can't handle the last buffered tuples for some Operators > ------------------------------------------------------------------------------ > > Key: PIG-4876 > URL: https://issues.apache.org/jira/browse/PIG-4876 > Project: Pig > Issue Type: Sub-task > Components: spark > Reporter: Xianda Ke > Assignee: Xianda Ke > Fix For: spark-branch > > Attachments: PIG-4876.patch > > > Some Operators, such as MergeCogroup, Stream, CollectedGroup etc buffer some > input records to constitute the result tuples. The last result tuples are > buffered in the operator. These Operators need a flag to indicate the end of > input, so that they can flush and constitute their last tuples. > Currently, the flag 'parentPlan.endOfAllInput' is targeted for flushing the > buffered tuples in MR mode. But it does not work with OutputConsumeIterator > in Spark mode. -- This message was sent by Atlassian JIRA (v6.3.4#6332)