[
https://issues.apache.org/jira/browse/HCATALOG-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HCATALOG-494:
------------------------------------------
Attachment: HCATALOG-494.patch
The fix.
1. Clone the Configuration object before creating JobContext/TaskAttemptContext
instances.
2. Check before deleting characters from StringBuilder.
> MultiOutputFormat in 0.23 fails to setAliasConf() correctly.
> ------------------------------------------------------------
>
> Key: HCATALOG-494
> URL: https://issues.apache.org/jira/browse/HCATALOG-494
> Project: HCatalog
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.4.1
> Environment: Hadoop-0.23.1
> Reporter: Mithun Radhakrishnan
> Assignee: Mithun Radhakrishnan
> Fix For: 0.4.1
>
> Attachments: HCATALOG-494.patch
>
>
> When using MultiOutputFormat (in Sta^H^H^H a super-duper top secret project
> that shall not be named), against Hadoop 0.23.1, one sees the following:
> <exception>
> java.lang.StringIndexOutOfBoundsException: String index out of range: -2
> at
> java.lang.AbstractStringBuilder.delete(AbstractStringBuilder.java:698)
> at java.lang.StringBuilder.delete(StringBuilder.java:255)
> at
> org.apache.hcatalog.mapreduce.MultiOutputFormat.setAliasConf(MultiOutputFormat.java:291)
> at
> org.apache.hcatalog.mapreduce.MultiOutputFormat.checkOutputSpecs(MultiOutputFormat.java:224)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:411)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:326)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1215)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1212)
> ...
> </exception>
> The error boils down to a change in the semantics of
> JobContext::JobContext(Configuration, JobID). While in .20, the Config was
> cloned, in .23 the Config is adopted (iff it's a JobConf). This causes the
> same Configuration instance to be written-to for different tables in the same
> job.
> An easy way to deal with this is to clone the Configuration before creating a
> JobContextImpl.
> A patch is on its way.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
