[
https://issues.apache.org/jira/browse/FLINK-1139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220776#comment-14220776
]
ASF GitHub Bot commented on FLINK-1139:
---------------------------------------
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/incubator-flink/pull/173#discussion_r20709141
--- Diff:
flink-addons/flink-hadoop-compatibility/src/main/java/org/apache/flink/hadoopcompatibility/mapred/HadoopOutputFormat.java
---
@@ -141,7 +142,20 @@ public void close() throws IOException {
if (this.fileOutputCommitter.needsTaskCommit(this.context)) {
this.fileOutputCommitter.commitTask(this.context);
}
- this.fileOutputCommitter.commitJob(this.jobContext);
+ }
+
+ @Override
+ public void finalizeGlobal(int parallelism) throws IOException {
+
+ try {
+ this.jobContext =
HadoopUtils.instantiateJobContext(this.jobConf, new JobID());
--- End diff --
This method is the last thing that runs from an operator. I would make all
variables stack variables, not fields of the class.
> HadoopOutputFormat fails for DOP > 1
> ------------------------------------
>
> Key: FLINK-1139
> URL: https://issues.apache.org/jira/browse/FLINK-1139
> Project: Flink
> Issue Type: Bug
> Components: Hadoop Compatibility
> Reporter: Fabian Hueske
> Assignee: Fabian Hueske
>
> HadoopOutputFormat fails for DOP > 1. The reason is that job finalization
> (removal of the temp directory) is done after the first task finishes.
> All other tasks will fail in that moment, since they cannot continue to write
> their data to the temp directory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
