[jira] [Closed] (TEZ-3030) java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/MRVersion
[ https://issues.apache.org/jira/browse/TEZ-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V closed TEZ-3030. > java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/MRVersion > -- > > Key: TEZ-3030 > URL: https://issues.apache.org/jira/browse/TEZ-3030 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: gewenhui > Fix For: 0.7.0 > > > hive (wzhl)> select count(1) from new_user; > Query ID = root_20160110223939_9394ddfa-e0d0-4bd2-abc4-44dc3c0d3e2a > Total jobs = 1 > Launching Job 1 out of 1 > Tez session was closed. Reopening... > Session re-established. > Status: Running (Executing on YARN cluster with App id > application_1449729554133_9102) > > VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED > KILLED > > Map 1 FAILED -1 00 -1 0 > 0 > Reducer 2 KILLED 1 001 0 > 0 > > VERTICES: 00/02 [>>--] 0%ELAPSED TIME: 0.22 s > > > Status: Failed > Vertex failed, vertexName=Map 1, vertexId=vertex_1449729554133_9102_1_00, > diagnostics=[Vertex vertex_1449729554133_9102_1_00 [Map 1] killed/failed due > to:ROOT_INPUT_INIT_FAILURE, Vertex Input: new_user initializer failed, > vertex=vertex_1449729554133_9102_1_00 [Map 1], > java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/MRVersion > at > org.apache.hadoop.hive.shims.Hadoop23Shims.isMR2(Hadoop23Shims.java:848) > at > org.apache.hadoop.hive.shims.Hadoop23Shims.getHadoopConfNames(Hadoop23Shims.java:919) > at > org.apache.hadoop.hive.conf.HiveConf$ConfVars.(HiveConf.java:356) > at > org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:371) > at > org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:296) > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:106) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:245) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:239) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:239) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:226) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.MRVersion > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > ... 17 more > ] > Vertex killed, vertexName=Reducer 2, vertexId=vertex_1449729554133_9102_1_01, > diagnostics=[Vertex received Kill in INITED state., Vertex > vertex_1449729554133_9102_1_01 [Reducer 2] killed/failed due to:null] > DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 > FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.tez.TezTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (TEZ-3031) java.lang.NoSuchMethodError
[ https://issues.apache.org/jira/browse/TEZ-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V closed TEZ-3031. > java.lang.NoSuchMethodError > --- > > Key: TEZ-3031 > URL: https://issues.apache.org/jira/browse/TEZ-3031 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 > Environment: CDH-5.4.8,hadoop-2.6.0,hive-1.1.0 >Reporter: gewenhui > Fix For: 0.7.0 > > > hive (wzhl)> set hive.execution.engine=tez; > hive (wzhl)> select count(1) from new_user; > Query ID = root_2016045454_76e6d365-df98-4a53-a8a5-8023e23865a9 > Total jobs = 1 > Launching Job 1 out of 1 > Status: Running (Executing on YARN cluster with App id > application_1449729554133_9383) > > VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED > KILLED > > Map 1 FAILED 2 002 7 > 1 > Reducer 2 KILLED 1 001 0 > 1 > > VERTICES: 00/02 [>>--] 0%ELAPSED TIME: 17.15 s > > > Status: Failed > Vertex failed, vertexName=Map 1, vertexId=vertex_1449729554133_9383_1_00, > diagnostics=[Task failed, taskId=task_1449729554133_9383_1_00_00, > diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running > task:java.lang.Exception: java.util.concurrent.ExecutionException: > java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID: method > (Ljava/lang/String;ILorg/apache/hadoop/mapreduce/TaskType;I)V not found > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:234) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.util.concurrent.ExecutionException: > java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID: method > (Ljava/lang/String;ILorg/apache/hadoop/mapreduce/TaskType;I)V not found > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:188) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:228) > ... 12 more > Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID: > method (Ljava/lang/String;ILorg/apache/hadoop/mapreduce/TaskType;I)V > not found > at > org.apache.tez.mapreduce.input.base.MRInputBase.initialize(MRInputBase.java:88) > at org.apache.tez.mapreduce.input.MRInput.initialize(MRInput.java:436) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeInputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:406) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeInputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:385) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > ... 4 more > ], TaskAttempt 1 failed, info=[Error: Failure while running > task:java.lang.Exception: java.util.concurrent.ExecutionException: > java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID: method > (Ljava/lang/String;ILorg/apache/hadoop/mapreduce/TaskType;I)V not found > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:234) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:17
[jira] [Resolved] (TEZ-3030) java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/MRVersion
[ https://issues.apache.org/jira/browse/TEZ-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gewenhui resolved TEZ-3030. --- Resolution: Fixed > java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/MRVersion > -- > > Key: TEZ-3030 > URL: https://issues.apache.org/jira/browse/TEZ-3030 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: gewenhui > Fix For: 0.7.0 > > > hive (wzhl)> select count(1) from new_user; > Query ID = root_20160110223939_9394ddfa-e0d0-4bd2-abc4-44dc3c0d3e2a > Total jobs = 1 > Launching Job 1 out of 1 > Tez session was closed. Reopening... > Session re-established. > Status: Running (Executing on YARN cluster with App id > application_1449729554133_9102) > > VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED > KILLED > > Map 1 FAILED -1 00 -1 0 > 0 > Reducer 2 KILLED 1 001 0 > 0 > > VERTICES: 00/02 [>>--] 0%ELAPSED TIME: 0.22 s > > > Status: Failed > Vertex failed, vertexName=Map 1, vertexId=vertex_1449729554133_9102_1_00, > diagnostics=[Vertex vertex_1449729554133_9102_1_00 [Map 1] killed/failed due > to:ROOT_INPUT_INIT_FAILURE, Vertex Input: new_user initializer failed, > vertex=vertex_1449729554133_9102_1_00 [Map 1], > java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/MRVersion > at > org.apache.hadoop.hive.shims.Hadoop23Shims.isMR2(Hadoop23Shims.java:848) > at > org.apache.hadoop.hive.shims.Hadoop23Shims.getHadoopConfNames(Hadoop23Shims.java:919) > at > org.apache.hadoop.hive.conf.HiveConf$ConfVars.(HiveConf.java:356) > at > org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:371) > at > org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:296) > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:106) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:245) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:239) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:239) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:226) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.MRVersion > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > ... 17 more > ] > Vertex killed, vertexName=Reducer 2, vertexId=vertex_1449729554133_9102_1_01, > diagnostics=[Vertex received Kill in INITED state., Vertex > vertex_1449729554133_9102_1_01 [Reducer 2] killed/failed due to:null] > DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1 > FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.tez.TezTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TEZ-3031) java.lang.NoSuchMethodError
[ https://issues.apache.org/jira/browse/TEZ-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gewenhui resolved TEZ-3031. --- Resolution: Fixed > java.lang.NoSuchMethodError > --- > > Key: TEZ-3031 > URL: https://issues.apache.org/jira/browse/TEZ-3031 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 > Environment: CDH-5.4.8,hadoop-2.6.0,hive-1.1.0 >Reporter: gewenhui > Fix For: 0.7.0 > > > hive (wzhl)> set hive.execution.engine=tez; > hive (wzhl)> select count(1) from new_user; > Query ID = root_2016045454_76e6d365-df98-4a53-a8a5-8023e23865a9 > Total jobs = 1 > Launching Job 1 out of 1 > Status: Running (Executing on YARN cluster with App id > application_1449729554133_9383) > > VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED > KILLED > > Map 1 FAILED 2 002 7 > 1 > Reducer 2 KILLED 1 001 0 > 1 > > VERTICES: 00/02 [>>--] 0%ELAPSED TIME: 17.15 s > > > Status: Failed > Vertex failed, vertexName=Map 1, vertexId=vertex_1449729554133_9383_1_00, > diagnostics=[Task failed, taskId=task_1449729554133_9383_1_00_00, > diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running > task:java.lang.Exception: java.util.concurrent.ExecutionException: > java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID: method > (Ljava/lang/String;ILorg/apache/hadoop/mapreduce/TaskType;I)V not found > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:234) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.util.concurrent.ExecutionException: > java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID: method > (Ljava/lang/String;ILorg/apache/hadoop/mapreduce/TaskType;I)V not found > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:188) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:228) > ... 12 more > Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID: > method (Ljava/lang/String;ILorg/apache/hadoop/mapreduce/TaskType;I)V > not found > at > org.apache.tez.mapreduce.input.base.MRInputBase.initialize(MRInputBase.java:88) > at org.apache.tez.mapreduce.input.MRInput.initialize(MRInput.java:436) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeInputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:406) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeInputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:385) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > ... 4 more > ], TaskAttempt 1 failed, info=[Error: Failure while running > task:java.lang.Exception: java.util.concurrent.ExecutionException: > java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID: method > (Ljava/lang/String;ILorg/apache/hadoop/mapreduce/TaskType;I)V not found > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:234) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable
[jira] [Commented] (TEZ-3031) java.lang.NoSuchMethodError
[ https://issues.apache.org/jira/browse/TEZ-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095563#comment-15095563 ] gewenhui commented on TEZ-3031: --- I don't open the url,but i resolved this problem.I add the hadoop-mapreduce-client-core-2.6.0-cdh5.4.8.jar to $HIVE_AUX_JARS_PATH > java.lang.NoSuchMethodError > --- > > Key: TEZ-3031 > URL: https://issues.apache.org/jira/browse/TEZ-3031 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.7.0 > Environment: CDH-5.4.8,hadoop-2.6.0,hive-1.1.0 >Reporter: gewenhui > Fix For: 0.7.0 > > > hive (wzhl)> set hive.execution.engine=tez; > hive (wzhl)> select count(1) from new_user; > Query ID = root_2016045454_76e6d365-df98-4a53-a8a5-8023e23865a9 > Total jobs = 1 > Launching Job 1 out of 1 > Status: Running (Executing on YARN cluster with App id > application_1449729554133_9383) > > VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED > KILLED > > Map 1 FAILED 2 002 7 > 1 > Reducer 2 KILLED 1 001 0 > 1 > > VERTICES: 00/02 [>>--] 0%ELAPSED TIME: 17.15 s > > > Status: Failed > Vertex failed, vertexName=Map 1, vertexId=vertex_1449729554133_9383_1_00, > diagnostics=[Task failed, taskId=task_1449729554133_9383_1_00_00, > diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running > task:java.lang.Exception: java.util.concurrent.ExecutionException: > java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID: method > (Ljava/lang/String;ILorg/apache/hadoop/mapreduce/TaskType;I)V not found > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:234) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171) > at > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.util.concurrent.ExecutionException: > java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID: method > (Ljava/lang/String;ILorg/apache/hadoop/mapreduce/TaskType;I)V not found > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:188) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:228) > ... 12 more > Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID: > method (Ljava/lang/String;ILorg/apache/hadoop/mapreduce/TaskType;I)V > not found > at > org.apache.tez.mapreduce.input.base.MRInputBase.initialize(MRInputBase.java:88) > at org.apache.tez.mapreduce.input.MRInput.initialize(MRInput.java:436) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeInputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:406) > at > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$InitializeInputCallable.callInternal(LogicalIOProcessorRuntimeTask.java:385) > at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > ... 4 more > ], TaskAttempt 1 failed, info=[Error: Failure while running > task:java.lang.Exception: java.util.concurrent.ExecutionException: > java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskID: method > (Ljava/lang/String;ILorg/apache/hadoop/mapreduce/TaskType;I)V not found > at > org.apache.t
[jira] [Commented] (TEZ-3034) YARN ATS: ATS leveldb fails to come up
[ https://issues.apache.org/jira/browse/TEZ-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095526#comment-15095526 ] Gopal V commented on TEZ-3034: -- [~llppll]: this is a YARN timeline server issue, moving the ticket to the right project. > YARN ATS: ATS leveldb fails to come up > -- > > Key: TEZ-3034 > URL: https://issues.apache.org/jira/browse/TEZ-3034 > Project: Apache Tez > Issue Type: Bug > Components: timelineserver > Environment: hadoop2.6.0+tez0.7.0 >Reporter: Lvpenglin > > My environment is hadoop2.6.0+tez0.7.0,when I start yarn timelineserver > occurred this problem,I can not solve it ,can you help me or give some > suggestion? Thank you very much. > 16/01/12 21:37:53 INFO service.AbstractService: Service > org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore failed in state > INITED; cause: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO > error: > /home/td/file:/home/td/temp/yarn/timeline/leveldb-timeline-store.ldb/LOCK: No > such file or directory > org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: > /home/td/file:/home/td/temp/yarn/timeline/leveldb-timeline-store.ldb/LOCK: No > such file or directory > at > org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:200) > at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:218) > at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:168) > at > org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore.serviceInit(LeveldbTimelineStore.java:219) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) > at > org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceInit(ApplicationHistoryServer.java:99) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.launchAppHistoryServer(ApplicationHistoryServer.java:157) > at > org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.main(ApplicationHistoryServer.java:167) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3034) YARN ATS: ATS leveldb fails to come up
[ https://issues.apache.org/jira/browse/TEZ-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3034: - Summary: YARN ATS: ATS leveldb fails to come up (was: set tez-ui occured problem) > YARN ATS: ATS leveldb fails to come up > -- > > Key: TEZ-3034 > URL: https://issues.apache.org/jira/browse/TEZ-3034 > Project: Apache Tez > Issue Type: Bug > Components: UI > Environment: hadoop2.6.0+tez0.7.0 >Reporter: Lvpenglin > > My environment is hadoop2.6.0+tez0.7.0,when I start yarn timelineserver > occurred this problem,I can not solve it ,can you help me or give some > suggestion? Thank you very much. > 16/01/12 21:37:53 INFO service.AbstractService: Service > org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore failed in state > INITED; cause: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO > error: > /home/td/file:/home/td/temp/yarn/timeline/leveldb-timeline-store.ldb/LOCK: No > such file or directory > org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: > /home/td/file:/home/td/temp/yarn/timeline/leveldb-timeline-store.ldb/LOCK: No > such file or directory > at > org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:200) > at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:218) > at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:168) > at > org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore.serviceInit(LeveldbTimelineStore.java:219) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) > at > org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceInit(ApplicationHistoryServer.java:99) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.launchAppHistoryServer(ApplicationHistoryServer.java:157) > at > org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.main(ApplicationHistoryServer.java:167) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3034) set tez-ui occured problem
Lvpenglin created TEZ-3034: -- Summary: set tez-ui occured problem Key: TEZ-3034 URL: https://issues.apache.org/jira/browse/TEZ-3034 Project: Apache Tez Issue Type: Bug Components: UI Environment: hadoop2.6.0+tez0.7.0 Reporter: Lvpenglin My environment is hadoop2.6.0+tez0.7.0,when I start yarn timelineserver occurred this problem,I can not solve it ,can you help me or give some suggestion? Thank you very much. 16/01/12 21:37:53 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore failed in state INITED; cause: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /home/td/file:/home/td/temp/yarn/timeline/leveldb-timeline-store.ldb/LOCK: No such file or directory org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /home/td/file:/home/td/temp/yarn/timeline/leveldb-timeline-store.ldb/LOCK: No such file or directory at org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:200) at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:218) at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:168) at org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore.serviceInit(LeveldbTimelineStore.java:219) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceInit(ApplicationHistoryServer.java:99) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.launchAppHistoryServer(ApplicationHistoryServer.java:157) at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.main(ApplicationHistoryServer.java:167) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3033) changes for 0.8.2 release
[ https://issues.apache.org/jira/browse/TEZ-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3033: Attachment: TEZ-3033.part1.txt Patch to update the version in master to 0.8.3-SNAPSHOT. > changes for 0.8.2 release > - > > Key: TEZ-3033 > URL: https://issues.apache.org/jira/browse/TEZ-3033 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-3033.part1.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-3033) changes for 0.8.2 release
Siddharth Seth created TEZ-3033: --- Summary: changes for 0.8.2 release Key: TEZ-3033 URL: https://issues.apache.org/jira/browse/TEZ-3033 Project: Apache Tez Issue Type: Task Reporter: Siddharth Seth Assignee: Siddharth Seth -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2865) Reduce default timeout for containers and tasks
[ https://issues.apache.org/jira/browse/TEZ-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-2865: Target Version/s: 0.8.3 (was: 0.8.2) > Reduce default timeout for containers and tasks > --- > > Key: TEZ-2865 > URL: https://issues.apache.org/jira/browse/TEZ-2865 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth > Attachments: TEZ-2865.1.txt > > > Currently set to 5 minutes, which is very long. A minute or less should be > sufficient to recover from intermittent issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3024) Move TaskCommunicator to correct package
[
https://issues.apache.org/jira/browse/TEZ-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095310#comment-15095310
]
TezQA commented on TEZ-3024:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12781943/TEZ-3024.1.jenkins.txt
against master revision 1d76543.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 10 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1421//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1421//console
This message is automatically generated.
> Move TaskCommunicator to correct package
>
>
> Key: TEZ-3024
> URL: https://issues.apache.org/jira/browse/TEZ-3024
> Project: Apache Tez
> Issue Type: Task
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: TEZ-3024.1.jenkins.txt, TEZ-3024.1.txt
>
>
> Currently under org.apache.tez.dag.api. Should be under
> org.apache.tez.serviceplugins.api
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Success: TEZ-3024 PreCommit Build #1421
Jira: https://issues.apache.org/jira/browse/TEZ-3024
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1421/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3784 lines...]
[INFO] Tez ... SUCCESS [ 0.024 s]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 53:08 min
[INFO] Finished at: 2016-01-13T00:39:12+00:00
[INFO] Final Memory: 78M/1171M
[INFO]
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12781943/TEZ-3024.1.jenkins.txt
against master revision 1d76543.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 10 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1421//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1421//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
be225f66d84159c89a9bfd4fa9b0f6939d9e0367 logged out
==
==
Finished build.
==
==
Archiving artifacts
[description-setter] Description set: TEZ-3024
Recording test results
Email was triggered for: Success
Sending email for trigger: Success
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Created] (TEZ-3032) DAG start time getting logged using system time instead of recorded time in startTime field
Hitesh Shah created TEZ-3032: Summary: DAG start time getting logged using system time instead of recorded time in startTime field Key: TEZ-3032 URL: https://issues.apache.org/jira/browse/TEZ-3032 Project: Apache Tez Issue Type: Bug Reporter: Hitesh Shah Priority: Minor This results in the start time getting modified when the finish event is logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2865) Reduce default timeout for containers and tasks
[ https://issues.apache.org/jira/browse/TEZ-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095235#comment-15095235 ] Siddharth Seth commented on TEZ-2865: - Is this good to go ? Not critical to get this done for 0.8.2 > Reduce default timeout for containers and tasks > --- > > Key: TEZ-2865 > URL: https://issues.apache.org/jira/browse/TEZ-2865 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth > Attachments: TEZ-2865.1.txt > > > Currently set to 5 minutes, which is very long. A minute or less should be > sufficient to recover from intermittent issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-3024) Move TaskCommunicator to correct package
[ https://issues.apache.org/jira/browse/TEZ-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095216#comment-15095216 ] Hitesh Shah commented on TEZ-3024: -- +1 pending pre-commit. > Move TaskCommunicator to correct package > > > Key: TEZ-3024 > URL: https://issues.apache.org/jira/browse/TEZ-3024 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-3024.1.jenkins.txt, TEZ-3024.1.txt > > > Currently under org.apache.tez.dag.api. Should be under > org.apache.tez.serviceplugins.api -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3024) Move TaskCommunicator to correct package
[ https://issues.apache.org/jira/browse/TEZ-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3024: Attachment: TEZ-3024.1.txt Patch for jenkins. > Move TaskCommunicator to correct package > > > Key: TEZ-3024 > URL: https://issues.apache.org/jira/browse/TEZ-3024 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-3024.1.jenkins.txt, TEZ-3024.1.txt > > > Currently under org.apache.tez.dag.api. Should be under > org.apache.tez.serviceplugins.api -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3024) Move TaskCommunicator to correct package
[ https://issues.apache.org/jira/browse/TEZ-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3024: Attachment: TEZ-3024.1.jenkins.txt Correct patch for jenkins. > Move TaskCommunicator to correct package > > > Key: TEZ-3024 > URL: https://issues.apache.org/jira/browse/TEZ-3024 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-3024.1.jenkins.txt, TEZ-3024.1.txt > > > Currently under org.apache.tez.dag.api. Should be under > org.apache.tez.serviceplugins.api -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3024) Move TaskCommunicator to correct package
[ https://issues.apache.org/jira/browse/TEZ-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3024: Attachment: (was: TEZ-3024.1.txt) > Move TaskCommunicator to correct package > > > Key: TEZ-3024 > URL: https://issues.apache.org/jira/browse/TEZ-3024 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-3024.1.jenkins.txt, TEZ-3024.1.txt > > > Currently under org.apache.tez.dag.api. Should be under > org.apache.tez.serviceplugins.api -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-3024) Move TaskCommunicator to correct package
[ https://issues.apache.org/jira/browse/TEZ-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-3024: Attachment: TEZ-3024.1.txt Patch for review. IDE refactor to move TaskCommunicator and related classes to org.apache.tez.serviceplugins.api [~hitesh] - please review. > Move TaskCommunicator to correct package > > > Key: TEZ-3024 > URL: https://issues.apache.org/jira/browse/TEZ-3024 > Project: Apache Tez > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-3024.1.jenkins.txt, TEZ-3024.1.txt > > > Currently under org.apache.tez.dag.api. Should be under > org.apache.tez.serviceplugins.api -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2669) Propagation of errors from plugins to the AM for error reporting
[ https://issues.apache.org/jira/browse/TEZ-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095188#comment-15095188 ] Siddharth Seth commented on TEZ-2669: - Thanks everyone for the reviews. Committing. Test failures are unrelated. > Propagation of errors from plugins to the AM for error reporting > > > Key: TEZ-2669 > URL: https://issues.apache.org/jira/browse/TEZ-2669 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.0-alpha >Reporter: Siddharth Seth >Assignee: Siddharth Seth >Priority: Blocker > Attachments: TEZ-2669.1.txt, TEZ-2669.2.txt, TEZ-2669.3.txt, > TEZ-2669.4.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2978) Add an option to allow the SplitGrouper to generate node local only groups
[ https://issues.apache.org/jira/browse/TEZ-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095187#comment-15095187 ] Siddharth Seth commented on TEZ-2978: - Committed to master. Test failures are unrelated. Thanks for the review. > Add an option to allow the SplitGrouper to generate node local only groups > -- > > Key: TEZ-2978 > URL: https://issues.apache.org/jira/browse/TEZ-2978 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Fix For: 0.8.2 > > Attachments: TEZ-2978.1.txt, TEZ-2978.2.txt > > > In certain cases, allowing the last few splits to be 'small' - smaller than > the requested size is beneficial, and can end up creating fewer splits. Also > this results in all splits having some form of locality information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2978) Add an option to allow the SplitGrouper to generate node local only groups
[ https://issues.apache.org/jira/browse/TEZ-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-2978: Summary: Add an option to allow the SplitGrouper to generate node local only groups (was: Add an option to allow small splits early on in TezSplitGrouper) > Add an option to allow the SplitGrouper to generate node local only groups > -- > > Key: TEZ-2978 > URL: https://issues.apache.org/jira/browse/TEZ-2978 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2978.1.txt, TEZ-2978.2.txt > > > In certain cases, allowing the last few splits to be 'small' - smaller than > the requested size is beneficial, and can end up creating fewer splits. Also > this results in all splits having some form of locality information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2669) Propagation of errors from plugins to the AM for error reporting
[ https://issues.apache.org/jira/browse/TEZ-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095184#comment-15095184 ] Hitesh Shah commented on TEZ-2669: -- +1 > Propagation of errors from plugins to the AM for error reporting > > > Key: TEZ-2669 > URL: https://issues.apache.org/jira/browse/TEZ-2669 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.0-alpha >Reporter: Siddharth Seth >Assignee: Siddharth Seth >Priority: Blocker > Attachments: TEZ-2669.1.txt, TEZ-2669.2.txt, TEZ-2669.3.txt, > TEZ-2669.4.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2669) Propagation of errors from plugins to the AM for error reporting
[
https://issues.apache.org/jira/browse/TEZ-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095179#comment-15095179
]
TezQA commented on TEZ-2669:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12781929/TEZ-2669.4.txt
against master revision 85637c6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 21 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.test.TestFaultTolerance
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1420//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1420//console
This message is automatically generated.
> Propagation of errors from plugins to the AM for error reporting
>
>
> Key: TEZ-2669
> URL: https://issues.apache.org/jira/browse/TEZ-2669
> Project: Apache Tez
> Issue Type: Sub-task
>Affects Versions: 0.8.0-alpha
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
>Priority: Blocker
> Attachments: TEZ-2669.1.txt, TEZ-2669.2.txt, TEZ-2669.3.txt,
> TEZ-2669.4.txt
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: TEZ-2669 PreCommit Build #1420
Jira: https://issues.apache.org/jira/browse/TEZ-2669
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1420/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3659 lines...]
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] [Help 2]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tez-tests
[INFO] Build failures were ignored.
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12781929/TEZ-2669.4.txt
against master revision 85637c6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 21 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.test.TestFaultTolerance
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1420//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1420//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
d290771e7d6ba497c68645c4587600cf10f880ee logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
7 tests failed.
FAILED: org.apache.tez.test.TestFaultTolerance.testRandomFailingInputs
Error Message:
expected: but was:
Stack Trace:
java.lang.AssertionError: expected: but was:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:141)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:124)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:120)
at
org.apache.tez.test.TestFaultTolerance.testRandomFailingInputs(TestFaultTolerance.java:763)
FAILED: org.apache.tez.test.TestFaultTolerance.testBasicInputFailureWithExit
Error Message:
TezSession has already shutdown. No cluster diagnostics found.
Stack Trace:
org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. No
cluster diagnostics found.
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:784)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:129)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:124)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:120)
at
org.apache.tez.test.TestFaultTolerance.testBasicInputFailureWithExit(TestFaultTolerance.java:261)
FAILED:
org.apache.tez.test.TestFaultTolerance.testInputFailureRerunCanSendOutputToTwoDownstreamVertices
Error Message:
TezSession has already shutdown. No cluster diagnostics found.
Stack Trace:
org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. No
cluster diagnostics found.
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:784)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:129)
at
org.apache.tez.test.TestFaultTolerance.runDAGA
[jira] [Updated] (TEZ-2669) Propagation of errors from plugins to the AM for error reporting
[ https://issues.apache.org/jira/browse/TEZ-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-2669: Attachment: TEZ-2669.4.txt Updated patch with a custom exception added to to the method signature. Rest of the patch remains the same. Haven't changed the unit tests which still catch RuntimeException. > Propagation of errors from plugins to the AM for error reporting > > > Key: TEZ-2669 > URL: https://issues.apache.org/jira/browse/TEZ-2669 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.0-alpha >Reporter: Siddharth Seth >Assignee: Siddharth Seth >Priority: Blocker > Attachments: TEZ-2669.1.txt, TEZ-2669.2.txt, TEZ-2669.3.txt, > TEZ-2669.4.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: TEZ-2978 PreCommit Build #1419
Jira: https://issues.apache.org/jira/browse/TEZ-2978
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1419/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3611 lines...]
[ERROR] [Help 2]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tez-tests
[INFO] Build failures were ignored.
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12781891/TEZ-2978.2.txt
against master revision 85637c6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.test.TestFaultTolerance
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1419//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1419//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
9cb73c3a7bfff7e77c1aa9e5ee31ce459a995318 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Compressed 3.17 MB of artifacts by 29.6% relative to #1405
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
###
## FAILED TESTS (if any)
##
7 tests failed.
FAILED: org.apache.tez.test.TestFaultTolerance.testRandomFailingInputs
Error Message:
expected: but was:
Stack Trace:
java.lang.AssertionError: expected: but was:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:141)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:124)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:120)
at
org.apache.tez.test.TestFaultTolerance.testRandomFailingInputs(TestFaultTolerance.java:763)
FAILED: org.apache.tez.test.TestFaultTolerance.testBasicInputFailureWithExit
Error Message:
TezSession has already shutdown. No cluster diagnostics found.
Stack Trace:
org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. No
cluster diagnostics found.
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:784)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:129)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:124)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:120)
at
org.apache.tez.test.TestFaultTolerance.testBasicInputFailureWithExit(TestFaultTolerance.java:261)
FAILED:
org.apache.tez.test.TestFaultTolerance.testInputFailureRerunCanSendOutputToTwoDownstreamVertices
Error Message:
TezSession has already shutdown. No cluster diagnostics found.
Stack Trace:
org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. No
cluster diagnostics found.
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:784)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.java:129)
at
org.apache.tez.test.TestFaultTolerance.runDAGAndVerify(TestFaultTolerance.ja
[jira] [Commented] (TEZ-2978) Add an option to allow small splits early on in TezSplitGrouper
[
https://issues.apache.org/jira/browse/TEZ-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094994#comment-15094994
]
TezQA commented on TEZ-2978:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12781891/TEZ-2978.2.txt
against master revision 85637c6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.test.TestFaultTolerance
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/1419//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1419//console
This message is automatically generated.
> Add an option to allow small splits early on in TezSplitGrouper
> ---
>
> Key: TEZ-2978
> URL: https://issues.apache.org/jira/browse/TEZ-2978
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: TEZ-2978.1.txt, TEZ-2978.2.txt
>
>
> In certain cases, allowing the last few splits to be 'small' - smaller than
> the requested size is beneficial, and can end up creating fewer splits. Also
> this results in all splits having some form of locality information.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2669) Propagation of errors from plugins to the AM for error reporting
[ https://issues.apache.org/jira/browse/TEZ-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094968#comment-15094968 ] Bikas Saha commented on TEZ-2669: - ServicePluginException sounds good. > Propagation of errors from plugins to the AM for error reporting > > > Key: TEZ-2669 > URL: https://issues.apache.org/jira/browse/TEZ-2669 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.0-alpha >Reporter: Siddharth Seth >Assignee: Siddharth Seth >Priority: Blocker > Attachments: TEZ-2669.1.txt, TEZ-2669.2.txt, TEZ-2669.3.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2669) Propagation of errors from plugins to the AM for error reporting
[ https://issues.apache.org/jira/browse/TEZ-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094573#comment-15094573 ] Hitesh Shah commented on TEZ-2669: -- [~bikassaha] any comments? > Propagation of errors from plugins to the AM for error reporting > > > Key: TEZ-2669 > URL: https://issues.apache.org/jira/browse/TEZ-2669 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.0-alpha >Reporter: Siddharth Seth >Assignee: Siddharth Seth >Priority: Blocker > Attachments: TEZ-2669.1.txt, TEZ-2669.2.txt, TEZ-2669.3.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2669) Propagation of errors from plugins to the AM for error reporting
[ https://issues.apache.org/jira/browse/TEZ-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094568#comment-15094568 ] Siddharth Seth commented on TEZ-2669: - bq. Thinking about this a bit more, can we consider declaring a checked exception along the lines of say "throws ServiceFatalException" ? This could be derived from either generic Exception or TezException. This would force plugin impls to only throw this declared exception in case of fatal errors. The exception handling code does not need to change i.e. should still catch Exception to account for unchecked exceptions thrown from plugins. Makes sense. Think this works better actually - since users are still forced to handle internal exceptions, and have an indication of what to throw. Will make the relevant changes. > Propagation of errors from plugins to the AM for error reporting > > > Key: TEZ-2669 > URL: https://issues.apache.org/jira/browse/TEZ-2669 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.0-alpha >Reporter: Siddharth Seth >Assignee: Siddharth Seth >Priority: Blocker > Attachments: TEZ-2669.1.txt, TEZ-2669.2.txt, TEZ-2669.3.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2978) Add an option to allow small splits early on in TezSplitGrouper
[ https://issues.apache.org/jira/browse/TEZ-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-2978: Attachment: TEZ-2978.2.txt Updated patch with renames. > Add an option to allow small splits early on in TezSplitGrouper > --- > > Key: TEZ-2978 > URL: https://issues.apache.org/jira/browse/TEZ-2978 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2978.1.txt, TEZ-2978.2.txt > > > In certain cases, allowing the last few splits to be 'small' - smaller than > the requested size is beneficial, and can end up creating fewer splits. Also > this results in all splits having some form of locality information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2669) Propagation of errors from plugins to the AM for error reporting
[ https://issues.apache.org/jira/browse/TEZ-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094510#comment-15094510 ] Hitesh Shah commented on TEZ-2669: -- Thinking about this a bit more, can we consider declaring a checked exception along the lines of say "throws ServiceFatalException" ? This could be derived from either generic Exception or TezException. This would force plugin impls to only throw this declared exception in case of fatal errors. The exception handling code does not need to change i.e. should still catch Exception to account for unchecked exceptions thrown from plugins. > Propagation of errors from plugins to the AM for error reporting > > > Key: TEZ-2669 > URL: https://issues.apache.org/jira/browse/TEZ-2669 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.0-alpha >Reporter: Siddharth Seth >Assignee: Siddharth Seth >Priority: Blocker > Attachments: TEZ-2669.1.txt, TEZ-2669.2.txt, TEZ-2669.3.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2669) Propagation of errors from plugins to the AM for error reporting
[ https://issues.apache.org/jira/browse/TEZ-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094443#comment-15094443 ] Bikas Saha commented on TEZ-2669: - Implementations should think about which exception matter to them and which dont. Trying to motivate them to think about exceptions by not having it in the API signature seems overdone (given the additional unnecessary code being added to actually have exceptions after all internally). It would be nice to hear from other reviewers what they think about this choice. If others are ok with it then we could commit this. Else it probably needs some review. So I defer to the comments of other reviewers. > Propagation of errors from plugins to the AM for error reporting > > > Key: TEZ-2669 > URL: https://issues.apache.org/jira/browse/TEZ-2669 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.0-alpha >Reporter: Siddharth Seth >Assignee: Siddharth Seth >Priority: Blocker > Attachments: TEZ-2669.1.txt, TEZ-2669.2.txt, TEZ-2669.3.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2669) Propagation of errors from plugins to the AM for error reporting
[ https://issues.apache.org/jira/browse/TEZ-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094121#comment-15094121 ] Hitesh Shah commented on TEZ-2669: -- Unused import related to org.apache.avro.reflect.Nullable. > Propagation of errors from plugins to the AM for error reporting > > > Key: TEZ-2669 > URL: https://issues.apache.org/jira/browse/TEZ-2669 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.0-alpha >Reporter: Siddharth Seth >Assignee: Siddharth Seth >Priority: Blocker > Attachments: TEZ-2669.1.txt, TEZ-2669.2.txt, TEZ-2669.3.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2669) Propagation of errors from plugins to the AM for error reporting
[ https://issues.apache.org/jira/browse/TEZ-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093505#comment-15093505 ] Siddharth Seth commented on TEZ-2669: - bq. The choice of not having Exception in the API's but then wrapping all plugins inside wrapper classes to add an exception to the effective internal signature seems incongruous. Simply having the Exception in the signature makes things clear and informs implementations that its ok to throw an exception and notify the framework vs forcing them to swallow any-and-all exceptions because the method signature does not allow throwing exceptions. It is different from signatures on other interfaces. However, I believe this model works better since users have to think about how to handle Exceptions - either catch and do something useful, or catch and throw a Runtime. The framework cannot do the first part about doing something useful. As an example - something like invokeRpc() throws IOException. This, with Exception on the plugin signature, could end up being invoked once and the exception trickles down to the framework. Without an exception on the plugin signature, the user needs to decide on what to do - which could be a retry. I've also heard from users that a 'generic throws Exception()' is not very useful. bq. Could you please remind me why TaskCommunicator.java is inside tez-dag instead of inside tez-api/serviceplugins like the rest of the plugins? And how does Hive/LLAP end up implementing it in that state? Primarily because of it's dependencies on IDs. Also things like TezEvent. All of this needs to change at some point in the future. LLAP depends on tez-dag to implement it's own TaskCommunicator. [~zjffdu], [~bikassaha], [~hitesh] - any other comments? > Propagation of errors from plugins to the AM for error reporting > > > Key: TEZ-2669 > URL: https://issues.apache.org/jira/browse/TEZ-2669 > Project: Apache Tez > Issue Type: Sub-task >Affects Versions: 0.8.0-alpha >Reporter: Siddharth Seth >Assignee: Siddharth Seth >Priority: Blocker > Attachments: TEZ-2669.1.txt, TEZ-2669.2.txt, TEZ-2669.3.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2978) Add an option to allow small splits early on in TezSplitGrouper
[ https://issues.apache.org/jira/browse/TEZ-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093499#comment-15093499 ] Siddharth Seth commented on TEZ-2978: - Thanks for the review. Makes sense to rename the property to be node-local-only. Will make the relevant config params, method etc renames and commit. > Add an option to allow small splits early on in TezSplitGrouper > --- > > Key: TEZ-2978 > URL: https://issues.apache.org/jira/browse/TEZ-2978 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: TEZ-2978.1.txt > > > In certain cases, allowing the last few splits to be 'small' - smaller than > the requested size is beneficial, and can end up creating fewer splits. Also > this results in all splits having some form of locality information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
