[
https://issues.apache.org/jira/browse/TEZ-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14490382#comment-14490382
]
Hitesh Shah commented on TEZ-2305:
----------------------------------
Comments:
Invalid import: import javax.xml.crypto.Data;
MR jobs when using yarn-tez mode do not use MROutputConfigBuilder. Plus they
also use MROutputLegacy
The change in the exception trace in TestMROutputConfigBuilder makes it seem
like we are changing current behavior that users might assume to be the default
behavior if something is not specified.
I dont think the new tests should worry about mapper only or not. It mainly is
a question of how a user should configure MROutput when using it against
mapred or mapreduce apis. And also, whether YARNRunner is doing the right thing
when building the dag and configuring MROutputLegacy correctly.
> MR compatibility sleep job fails with IOException: Undefined job output-path
> ----------------------------------------------------------------------------
>
> Key: TEZ-2305
> URL: https://issues.apache.org/jira/browse/TEZ-2305
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Tassapol Athiapinya
> Priority: Critical
> Attachments: TEZ-2305-3.patch, TEZ-2305-4.patch, TEZ-2305.1.patch,
> TEZ-2305.2.patch
>
>
> Running MR sleep job has an IOException.
> {code}
> 15/04/09 20:52:25 INFO mapreduce.Job: Job job_1428612196442_0002 failed with
> state FAILED due to: Vertex failed, vertexName=initialmap,
> vertexId=vertex_1428612196442_0002_1_00, diagnostics=[Task failed,
> taskId=task_1428612196442_0002_1_00_000001, diagnostics=[TaskAttempt 0
> failed, info=[Error: Failure while running task:java.io.IOException:
> Undefined job output-path
> at
> org.apache.hadoop.mapred.FileOutputFormat.getTaskOutputPath(FileOutputFormat.java:248)
> at
> org.apache.hadoop.mapred.TextOutputFormat.getRecordWriter(TextOutputFormat.java:121)
> at
> org.apache.tez.mapreduce.output.MROutput.initialize(MROutput.java:401)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:436)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:415)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> ], TaskAttempt 1 failed, info=[Error: Failure while running
> task:java.io.IOException: Undefined job output-path
> at
> org.apache.hadoop.mapred.FileOutputFormat.getTaskOutputPath(FileOutputFormat.java:248)
> at
> org.apache.hadoop.mapred.TextOutputFormat.getRecordWriter(TextOutputFormat.java:121)
> at
> org.apache.tez.mapreduce.output.MROutput.initialize(MROutput.java:401)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:436)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:415)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> ], TaskAttempt 2 failed, info=[Error: Failure while running
> task:java.io.IOException: Undefined job output-path
> at
> org.apache.hadoop.mapred.FileOutputFormat.getTaskOutputPath(FileOutputFormat.java:248)
> at
> org.apache.hadoop.mapred.TextOutputFormat.getRecordWriter(TextOutputFormat.java:121)
> at
> org.apache.tez.mapreduce.output.MROutput.initialize(MROutput.java:401)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:436)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:415)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> ], TaskAttempt 3 failed, info=[Error: Failure while running
> task:java.io.IOException: Undefined job output-path
> at
> org.apache.hadoop.mapred.FileOutputFormat.getTaskOutputPath(FileOutputFormat.java:248)
> at
> org.apache.hadoop.mapred.TextOutputFormat.getRecordWriter(TextOutputFormat.java:121)
> at
> org.apache.tez.mapreduce.output.MROutput.initialize(MROutput.java:401)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:436)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeOutputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:415)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> ]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex
> vertex_1428612196442_0002_1_00 [initialmap] killed/failed due to:null]. DAG
> failed due to vertex failure. failedVertices:1 killedVertices:0
> 15/04/09 20:52:25 INFO mapreduce.Job: Counters: 0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
