[
https://issues.apache.org/jira/browse/HCATALOG-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453703#comment-13453703
]
Rohini Palaniswamy commented on HCATALOG-499:
---------------------------------------------
Travis,
Mapred framework sets the mapred.work.output.dir (WorkOutputPath for
FileOutputComitter) to the output path if the OutputCommitter is not a
FileOutputComitter. Previously that code in mapreduce never had any effect as
it was being set in a copy of the JobConf. With 23, that is a reference and so
it gets set in TaskAttemptContext. If it is null, FileOutputComitter computes
it again. But with that set it looks at that location for the output during
commitTask and since the temporary files are not there, it throws an error.
The unit tests currently do not run when compiled in H23 mode. This requires
version of pig and hive that work with 23. Pig 0.10 works with 23, but we need
to upgrade antlr for that as you know. Also only the H20 version of pig is in
the maven repository(PIG-2907) which can soon be fixed. And Chris in our team
is close to getting hive to fully work with 23. Till we fix these issues, unit
tests can be run only against hadoop 20.
> Multiple store commands does not work with Hadoop23
> ---------------------------------------------------
>
> Key: HCATALOG-499
> URL: https://issues.apache.org/jira/browse/HCATALOG-499
> Project: HCatalog
> Issue Type: Bug
> Affects Versions: 0.4.1
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.5, 0.4.1
>
> Attachments: HCATALOG-499.patch
>
>
> There is change in the semantics of
> JobContext::JobContext(Configuration, JobID). While in .20, the Config was
> cloned, in .23 the Config is adopted (if it's a JobConf). That combined with
> the way mapred Task.java handles output committers that do not extend
> FileOutputCommitter has broken storing different partitions to the same table
> with multiple store statements in pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
