[jira] [Commented] (PIG-3741) Utils.setTmpFileCompressionOnConf can cause side effect for SequenceFileInterStorage
[ https://issues.apache.org/jira/browse/PIG-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13892731#comment-13892731 ] Aniket Mokashi commented on PIG-3741: - Committed to 0.12.1 branch and trunk. Thanks [~julienledem] for the review! > Utils.setTmpFileCompressionOnConf can cause side effect for > SequenceFileInterStorage > > > Key: PIG-3741 > URL: https://issues.apache.org/jira/browse/PIG-3741 > Project: Pig > Issue Type: Bug >Reporter: Aniket Mokashi >Assignee: Aniket Mokashi > Fix For: 0.12.1, 0.13.0 > > Attachments: PIG-3741.patch > > > Currently, Utils.setTmpFileCompressionOnConf(pigContext, conf); is invoked > for every job. In case of Seqfile, this api sets mapreduce params on conf to > assist SequenceFileInterStorage. However, as a side effect, this might change > the behavior of other storers due to these mapred properties. This api should > only be called for jobs with intermediate storage. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (PIG-3741) Utils.setTmpFileCompressionOnConf can cause side effect for SequenceFileInterStorage
[ https://issues.apache.org/jira/browse/PIG-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13891030#comment-13891030 ] Julien Le Dem commented on PIG-3741: Ideally each store would get its own config object, but that would be a major refactoring. In the meantime, this looks like a good improvement to me. +1 > Utils.setTmpFileCompressionOnConf can cause side effect for > SequenceFileInterStorage > > > Key: PIG-3741 > URL: https://issues.apache.org/jira/browse/PIG-3741 > Project: Pig > Issue Type: Bug >Reporter: Aniket Mokashi >Assignee: Aniket Mokashi > Fix For: 0.12.1 > > Attachments: PIG-3741.patch > > > Currently, Utils.setTmpFileCompressionOnConf(pigContext, conf); is invoked > for every job. In case of Seqfile, this api sets mapreduce params on conf to > assist SequenceFileInterStorage. However, as a side effect, this might change > the behavior of other storers due to these mapred properties. This api should > only be called for jobs with intermediate storage. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
