Mithun Radhakrishnan created HCATALOG-494:
---------------------------------------------
Summary: MultiOutputFormat in 0.23 fails to setAliasConf()
correctly.
Key: HCATALOG-494
URL: https://issues.apache.org/jira/browse/HCATALOG-494
Project: HCatalog
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.4.1
Environment: Hadoop-0.23.1
Reporter: Mithun Radhakrishnan
Fix For: 0.4.1
When using MultiOutputFormat (in Sta^H^H^H a super-duper top secret project
that shall not be named), against Hadoop 0.23.1, one sees the following:
<exception>
java.lang.StringIndexOutOfBoundsException: String index out of range: -2
at
java.lang.AbstractStringBuilder.delete(AbstractStringBuilder.java:698)
at java.lang.StringBuilder.delete(StringBuilder.java:255)
at
org.apache.hcatalog.mapreduce.MultiOutputFormat.setAliasConf(MultiOutputFormat.java:291)
at
org.apache.hcatalog.mapreduce.MultiOutputFormat.checkOutputSpecs(MultiOutputFormat.java:224)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:411)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:326)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1215)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1212)
...
</exception>
The error boils down to a change in the semantics of
JobContext::JobContext(Configuration, JobID). While in .20, the Config was
cloned, in .23 the Config is adopted (iff it's a JobConf). This causes the same
Configuration instance to be written-to for different tables in the same job.
An easy way to deal with this is to clone the Configuration before creating a
JobContextImpl.
A patch is on its way.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
