[
https://issues.apache.org/jira/browse/TEZ-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121295#comment-15121295
]
Lvpenglin commented on TEZ-3071:
--------------------------------
ok,thank you very much.I have some question want to ask for you,My tez program
needs to parse the config file,which is parsed by the dom4j just same as parse
the local file.I only set the file path to local path,when run the program on
the cluster,I wonder tez whether can load my local file to hdfs and share to
the slave to use.
> run tez task on local mode is success,but run on cluster occurred this
> problem,java.lang.NullPointerException at
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78)
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: TEZ-3071
> URL: https://issues.apache.org/jira/browse/TEZ-3071
> Project: Apache Tez
> Issue Type: Task
> Environment: hadoop2.6.0+tez0.7.0
> Reporter: Lvpenglin
>
> 16/01/24 21:29:13 INFO client.DAGClientImpl: Waiting for DAG to start running
> 16/01/24 21:29:25 INFO client.DAGClientImpl: DAG initialized:
> CurrentState=Running
> 16/01/24 21:29:26 INFO client.DAGClientImpl: DAG: State: RUNNING Progress: 0%
> TotalTasks: 2 Succeeded: 0 Running: 0 Failed: 0 Killed: 0
> 16/01/24 21:29:26 INFO client.DAGClientImpl: VertexStatus: VertexName:
> ASSOCIATE Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed: 0
> Killed: 0
> 16/01/24 21:29:26 INFO client.DAGClientImpl: VertexStatus: VertexName:
> SALES_TRANSACTION Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed:
> 0 Killed: 0
> 16/01/24 21:29:32 INFO client.DAGClientImpl: DAG: State: RUNNING Progress: 0%
> TotalTasks: 2 Succeeded: 0 Running: 0 Failed: 0 Killed: 0
> 16/01/24 21:29:32 INFO client.DAGClientImpl: VertexStatus: VertexName:
> ASSOCIATE Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed: 0
> Killed: 0
> 16/01/24 21:29:32 INFO client.DAGClientImpl: VertexStatus: VertexName:
> SALES_TRANSACTION Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed:
> 0 Killed: 0
> 16/01/24 21:29:37 INFO client.DAGClientImpl: DAG: State: RUNNING Progress: 0%
> TotalTasks: 2 Succeeded: 0 Running: 1 Failed: 0 Killed: 0
> 16/01/24 21:29:37 INFO client.DAGClientImpl: VertexStatus: VertexName:
> ASSOCIATE Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 1 Failed: 0
> Killed: 0
> 16/01/24 21:29:37 INFO client.DAGClientImpl: VertexStatus: VertexName:
> SALES_TRANSACTION Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed:
> 0 Killed: 0
> 16/01/24 21:29:42 INFO client.DAGClientImpl: DAG: State: RUNNING Progress: 0%
> TotalTasks: 2 Succeeded: 0 Running: 1 Failed: 0 Killed: 0 FailedTaskAttempts:
> 1
> 16/01/24 21:29:42 INFO client.DAGClientImpl: VertexStatus: VertexName:
> ASSOCIATE Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 1 Failed: 0
> Killed: 0 FailedTaskAttempts: 1
> 16/01/24 21:29:42 INFO client.DAGClientImpl: VertexStatus: VertexName:
> SALES_TRANSACTION Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed:
> 0 Killed: 0
> 16/01/24 21:29:47 INFO client.DAGClientImpl: DAG: State: RUNNING Progress: 0%
> TotalTasks: 2 Succeeded: 0 Running: 1 Failed: 0 Killed: 0 FailedTaskAttempts:
> 1
> 16/01/24 21:29:47 INFO client.DAGClientImpl: VertexStatus: VertexName:
> ASSOCIATE Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 1 Failed: 0
> Killed: 0 FailedTaskAttempts: 1
> 16/01/24 21:29:47 INFO client.DAGClientImpl: VertexStatus: VertexName:
> SALES_TRANSACTION Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed:
> 0 Killed: 0
> 16/01/24 21:29:52 INFO client.DAGClientImpl: DAG: State: RUNNING Progress: 0%
> TotalTasks: 2 Succeeded: 0 Running: 1 Failed: 0 Killed: 0 FailedTaskAttempts:
> 2
> 16/01/24 21:29:52 INFO client.DAGClientImpl: VertexStatus: VertexName:
> ASSOCIATE Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 1 Failed: 0
> Killed: 0 FailedTaskAttempts: 2
> 16/01/24 21:29:52 INFO client.DAGClientImpl: VertexStatus: VertexName:
> SALES_TRANSACTION Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed:
> 0 Killed: 0
> 16/01/24 21:29:57 INFO client.DAGClientImpl: DAG: State: RUNNING Progress: 0%
> TotalTasks: 2 Succeeded: 0 Running: 1 Failed: 0 Killed: 0 FailedTaskAttempts:
> 2
> 16/01/24 21:29:57 INFO client.DAGClientImpl: VertexStatus: VertexName:
> ASSOCIATE Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 1 Failed: 0
> Killed: 0 FailedTaskAttempts: 2
> 16/01/24 21:29:57 INFO client.DAGClientImpl: VertexStatus: VertexName:
> SALES_TRANSACTION Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed:
> 0 Killed: 0
> 16/01/24 21:30:02 INFO client.DAGClientImpl: DAG: State: RUNNING Progress: 0%
> TotalTasks: 2 Succeeded: 0 Running: 1 Failed: 0 Killed: 0 FailedTaskAttempts:
> 3
> 16/01/24 21:30:02 INFO client.DAGClientImpl: VertexStatus: VertexName:
> ASSOCIATE Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 1 Failed: 0
> Killed: 0 FailedTaskAttempts: 3
> 16/01/24 21:30:02 INFO client.DAGClientImpl: VertexStatus: VertexName:
> SALES_TRANSACTION Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed:
> 0 Killed: 0
> 16/01/24 21:30:04 INFO client.DAGClientImpl: DAG: State: FAILED Progress: 0%
> TotalTasks: 2 Succeeded: 0 Running: 0 Failed: 1 Killed: 1 FailedTaskAttempts:
> 4
> 16/01/24 21:30:04 INFO client.DAGClientImpl: VertexStatus: VertexName:
> ASSOCIATE Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed: 1
> Killed: 0 FailedTaskAttempts: 4
> 16/01/24 21:30:04 INFO client.DAGClientImpl: VertexStatus: VertexName:
> SALES_TRANSACTION Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed:
> 0 Killed: 1
> 16/01/24 21:30:04 INFO client.DAGClientImpl: DAG completed. FinalState=FAILED
> 16/01/24 21:30:04 INFO sgprocessor.SqllabTezBase: DAG diagnostics: [Vertex
> failed, vertexName=ASSOCIATE, vertexId=vertex_1453425211412_0006_1_00,
> diagnostics=[Task failed, taskId=task_1453425211412_0006_1_00_000000,
> diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running
> task:java.lang.NullPointerException
> at
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78)
> at
> org.apache.tez.runtime.library.processor.SimpleProcessor.run(SimpleProcessor.java:53)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> ], TaskAttempt 1 failed, info=[Error: Failure while running
> task:java.lang.NullPointerException
> at
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78)
> at
> org.apache.tez.runtime.library.processor.SimpleProcessor.run(SimpleProcessor.java:53)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> ], TaskAttempt 2 failed, info=[Error: Failure while running
> task:java.lang.NullPointerException
> at
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78)
> at
> org.apache.tez.runtime.library.processor.SimpleProcessor.run(SimpleProcessor.java:53)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> ], TaskAttempt 3 failed, info=[Error: Failure while running
> task:java.lang.NullPointerException
> at
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78)
> at
> org.apache.tez.runtime.library.processor.SimpleProcessor.run(SimpleProcessor.java:53)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:0, Vertex vertex_1453425211412_0006_1_00 [ASSOCIATE]
> killed/failed due to:null], Vertex killed, vertexName=SALES_TRANSACTION,
> vertexId=vertex_1453425211412_0006_1_01, diagnostics=[Vertex received Kill
> while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE,
> failedTasks:0 killedTasks:1, Vertex vertex_1453425211412_0006_1_01
> [SALES_TRANSACTION] killed/failed due to:null], DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
