GitHub user twalthr opened a pull request:
https://github.com/apache/incubator-flink/pull/75
Small fixes for running Hadoop OutputFormats with HadoopCompat layer
This PR includes some fixes necessary to run a large Hadoop OutputFormat
such as the HCatOutputFormat on HDFS. Since HCatOutputFormat uses a custom
output file name the following exception is thrown which is now fixed:
```java
eu.stratosphere.client.program.ProgramInvocationException: The program
execution failed: java.lang.IllegalArgumentException: This method can only be
called from within a Job
at
org.apache.hadoop.mapred.FileOutputFormat.getUniqueName(FileOutputFormat.java:286)
at
org.apache.hive.hcatalog.mapreduce.FileOutputFormatContainer.getRecordWriter(FileOutputFormatContainer.java:107)
at
org.apache.hive.hcatalog.mapreduce.HCatOutputFormat.getRecordWriter(HCatOutputFormat.java:246)
at
eu.stratosphere.hadoopcompatibility.mapreduce.HadoopOutputFormat.open(HadoopOutputFormat.java:122)
```
Furthermore there were some issues with HDFS that were also fixed with this
PR.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/twalthr/incubator-flink HadoopCompatHCatReady
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/75.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #75
----
commit 82926a19e71ce03ec404413682a4ffbba4590666
Author: twalthr <[email protected]>
Date: 2014-07-21T10:11:16Z
Small bug fixes for running hadoop output formats
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
