[
https://issues.apache.org/jira/browse/PIG-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495887#comment-13495887
]
Dmitriy V. Ryaboy commented on PIG-2553:
----------------------------------------
I think it should be set to false by default so that current scripts that use
fancy storers, etc, can keep running without change (the scripts that have
bugs, which we are trying to address with this, don't run correctly at all, so
we don't have to worry about being backwards compatible with them). Individual
pig admins / script authors can decide to turn it on by default if they notice
this happening a lot.
We are piling up quite the list of exceptions, though. Between hcat, hbase,
unknown other schemas, and hdfs/s3/kfs/mapr cases, I'm getting concerned that
maybe this wasn't such a well thought out feature wish on my part!
What do you guys think?
> Pig shouldn't allow attempts to write multiple relations into same directory
> ----------------------------------------------------------------------------
>
> Key: PIG-2553
> URL: https://issues.apache.org/jira/browse/PIG-2553
> Project: Pig
> Issue Type: Improvement
> Reporter: Dmitriy V. Ryaboy
> Assignee: Prashant Kommireddi
> Attachments: PIG-2553.patch
>
>
> We've seen multiple occasions where users accidentally try to store 2 or more
> different relations to the same destination directory. Currently, this passes
> the Pig planner and fails on MR side due to concurrent attempts to create the
> same part file on the reducer. This is extremely confusing to the user, and
> hard to debug.
> We should instead fail their scripts before they are even submitted, since we
> can identify the erroneous condition from the beginning.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
