[
https://issues.apache.org/jira/browse/HIVE-790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ning Zhang updated HIVE-790:
----------------------------
Attachment: Hive-790_2.patch
This patch incorporated Zheng's comments.
Also changed the UnionOperator.processOp to synchronized. This seems fixed the
race condition issue -- the 2nd mapping phase passed w/o IOException. The
reducers are still running but the "UNION" and "SCRIPT" operators should
already passed.
Will look into any possible performance hit of the introduction of
synchronized. We should open another JIRA if there is any.
> race condition related to ScriptOperator + UnionOperator
> --------------------------------------------------------
>
> Key: HIVE-790
> URL: https://issues.apache.org/jira/browse/HIVE-790
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Zheng Shao
> Assignee: Ning Zhang
> Attachments: Hive-790.patch, Hive-790_2.patch
>
>
> ScriptOperator uses a second thread to output the rows to the children
> operators. In a corner case which contains a union, 2 threads might be
> outputting data into the same operator hierarchy and caused race conditions.
> {code}
> CREATE TABLE tablea (cola STRING);
> SELECT *
> FROM (
> SELECT TRANSFORM(cola)
> USING 'cat'
> AS cola
> FROM tablea
> UNION ALL
> SELECT cola as cola
> FROM tablea
> ) a;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
