[ https://issues.apache.org/jira/browse/TEZ-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111778#comment-15111778 ]
Lvpenglin commented on TEZ-3055: -------------------------------- thank you very much.but when I set this property,have some error as the following.I want to ask how to set the tez program which can be run on my cluster,because I only found how to set the program run in local mode,I don't know if need some special set.Can you give some suggestion or data? Vertex failed, vertexName=ASSOCIATE, vertexId=vertex_1453425211412_0004_1_00, diagnostics= Task failed, taskId=task_1453425211412_0004_1_00_000000, diagnostics= TaskAttempt 0 failed, info= Error: Failure while running task:java.lang.NullPointerException at com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78) at org.apache.tez.runtime.library.processor.SimpleProcessor.run(SimpleProcessor.java:53) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) TaskAttempt 1 failed, info= Error: Failure while running task:java.lang.NullPointerException at com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78) at org.apache.tez.runtime.library.processor.SimpleProcessor.run(SimpleProcessor.java:53) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) TaskAttempt 2 failed, info= Error: Failure while running task:java.lang.NullPointerException at com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78) at org.apache.tez.runtime.library.processor.SimpleProcessor.run(SimpleProcessor.java:53) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) TaskAttempt 3 failed, info= Error: Failure while running task:java.lang.NullPointerException at com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor.run(TezDataGeneratorTest.java:78) at org.apache.tez.runtime.library.processor.SimpleProcessor.run(SimpleProcessor.java:53) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1453425211412_0004_1_00 ASSOCIATE killed/failed due to:null Vertex killed, vertexName=SALES_TRANSACTION, vertexId=vertex_1453425211412_0004_1_01, diagnostics= Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1453425211412_0004_1_01 SALES_TRANSACTION killed/failed due to:null DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 > I got this jar is by the mvn package and run it on my cluster,occurred this > problem.I can run the example which the tez given successful.But when I run > this jar which is packaged by mvn packsge myself is failed. > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: TEZ-3055 > URL: https://issues.apache.org/jira/browse/TEZ-3055 > Project: Apache Tez > Issue Type: Improvement > Environment: this is the tez orderedwordcount example,I want it can > be run on my cluster,but occurred this problem.This code is packaged by > myself.When I run the tez's example jar on my cluster,it's ok.My environment > is hadoop2.6.0+tez0.7.0.Can you give me some suggestion? > Reporter: Lvpenglin > Labels: jar, maven-package, run-example, tez > > Vertex failed, vertexName=ASSOCIATE, vertexId=vertex_1452667827400_0022_1_00, > diagnostics= > Task failed, taskId=task_1452667827400_0022_1_00_000000, diagnostics= > TaskAttempt 0 failed, info= > Error: Failure while running > task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45) > at > org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ClassNotFoundException: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:274) > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:43) > ... 15 more > TaskAttempt 1 failed, info= > Error: Failure while running > task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45) > at > org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ClassNotFoundException: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:274) > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:43) > ... 15 more > TaskAttempt 2 failed, info= > Error: Failure while running > task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45) > at > org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ClassNotFoundException: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:274) > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:43) > ... 15 more > TaskAttempt 3 failed, info= > Error: Failure while running > task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45) > at > org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ClassNotFoundException: > com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:274) > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:43) > ... 15 more > Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, > Vertex vertex_1452667827400_0022_1_00 > ASSOCIATE > killed/failed due to:null > Vertex killed, vertexName=SALES_TRANSACTION, > vertexId=vertex_1452667827400_0022_1_01, diagnostics= > Vertex received Kill while in RUNNING state., Vertex did not succeed due to > OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex > vertex_1452667827400_0022_1_01 > SALES_TRANSACTION > killed/failed due to:null > DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)