Please create a jira with these details and also a pig script with which we can reproduce the issue.
Thanks, Rohini On Fri, Mar 27, 2015 at 8:01 AM, Michael Howard <[email protected]> wrote: > I am a newbie. > Isn't clear to me if I should post this type of issue to dev or user, so I > decided to post to [email protected] > > We have a reproducible script that works with -x mapreduce, fails with -x > tez > > Failure while running > task:org.apache.pig.backend.executionengine.ExecException: ERROR 0: Input > from vertex scope-108 is missing > [snip] > pigstats.tez.TezPigScriptStats - Script Statistics: > > HadoopVersion: 2.5.0-cdh5.3.2 > PigVersion: 0.15.0-SNAPSHOT > TezVersion: 0.6.0 > UserId: root > FileName: foo.pig > StartedAt: 2015-03-27 08:47:40 > FinishedAt: 2015-03-27 08:49:08 > Features: HASH_JOIN,GROUP_BY,FILTER,UNION > > Failed! > > Q: Is this a known issue? > Q: What kind of diagnostics/info can I provide to assist? > > Michael > > > full backtrace > > , progress=TotalTasks: 497 Succeeded: 24 Running: 0 Failed: 1 Killed: 472 > FailedTaskAttempts: 11, diagnostics=Vertex failed, vertexName=scope-122, > vertexId=vertex_1427151599024_0195_1_08, diagnostics=[Task failed, > taskId=task_1427151599024_0195_1_08_000001, diagnostics=[TaskAttempt 0 > failed, info=[Error: Failure while running > task:org.apache.pig.backend.executionengine.ExecException: ERROR 0: Input > from vertex scope-108 is missing > at > > org.apache.pig.backend.hadoop.executionengine.tez.plan.udf.ReadScalarsTez.attachInputs(ReadScalarsTez.java:77) > at > > org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.initializeInputs(PigProcessor.java:293) > at > > org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:183) > at > > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324) > at > > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168) > at > > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > ], TaskAttempt 1 failed, info=[Error: Failure while running > task:org.apache.pig.backend.executionengine.ExecException: ERROR 0: Input > from vertex scope-108 is missing > at > > org.apache.pig.backend.hadoop.executionengine.tez.plan.udf.ReadScalarsTez.attachInputs(ReadScalarsTez.java:77) > at > > org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.initializeInputs(PigProcessor.java:293) > at > > org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:183) > at > > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324) > at > > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168) > at > > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > ], TaskAttempt 2 failed, info=[Error: Failure while running > task:org.apache.pig.backend.executionengine.ExecException: ERROR 0: Input > from vertex scope-108 is missing > at > > org.apache.pig.backend.hadoop.executionengine.tez.plan.udf.ReadScalarsTez.attachInputs(ReadScalarsTez.java:77) > at > > org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.initializeInputs(PigProcessor.java:293) > at > > org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:183) > at > > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324) > at > > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176) > at > > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168) > at > > org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > ], TaskAttempt 3 failed, info=[Container > container_1427151599024_0195_01_000010 finished with diagnostics set to > [Container preempted internally]]], Vertex failed as one or more tasks > failed. failedTasks:1, Vertex vertex_1427151599024_0195_1_08 [scope-122] > killed/failed due to:null] > Vertex killed, vertexName=scope-129, > vertexId=vertex_1427151599024_0195_1_09, diagnostics=[Vertex received Kill > while in RUNNING state., Vertex killed as other vertex failed. > failedTasks:0, Vertex vertex_1427151599024_0195_1_09 [scope-129] > killed/failed due to:null] > Vertex killed, vertexName=scope-130, > vertexId=vertex_1427151599024_0195_1_10, diagnostics=[Vertex received Kill > while in RUNNING state., Vertex killed as other vertex failed. > failedTasks:0, Vertex vertex_1427151599024_0195_1_10 [scope-130] > killed/failed due to:null] > Vertex killed, vertexName=scope-128, > vertexId=vertex_1427151599024_0195_1_07, diagnostics=[Vertex received Kill > while in RUNNING state., Vertex killed as other vertex failed. > failedTasks:0, Vertex vertex_1427151599024_0195_1_07 [scope-128] > killed/failed due to:null] > Vertex killed, vertexName=scope-126, > vertexId=vertex_1427151599024_0195_1_05, diagnostics=[Vertex received Kill > while in RUNNING state., Vertex killed as other vertex failed. > failedTasks:0, Vertex vertex_1427151599024_0195_1_05 [scope-126] > killed/failed due to:null] > Vertex killed, vertexName=scope-125, > vertexId=vertex_1427151599024_0195_1_06, diagnostics=[Vertex received Kill > while in RUNNING state., Vertex killed as other vertex failed. > failedTasks:0, Vertex vertex_1427151599024_0195_1_06 [scope-125] > killed/failed due to:null] > Vertex killed, vertexName=scope-115, > vertexId=vertex_1427151599024_0195_1_04, diagnostics=[Vertex received Kill > while in RUNNING state., Vertex killed as other vertex failed. > failedTasks:0, Vertex vertex_1427151599024_0195_1_04 [scope-115] > killed/failed due to:null] > DAG failed due to vertex failure. failedVertices:1 killedVertices:6, > counters=Counters: 43 > org.apache.tez.common.counters.DAGCounter > NUM_FAILED_TASKS=11 > NUM_KILLED_TASKS=36 > NUM_SUCCEEDED_TASKS=24 > TOTAL_LAUNCHED_TASKS=51 > OTHER_LOCAL_TASKS=1 > DATA_LOCAL_TASKS=21 > File System Counters > FILE_BYTES_READ=187910630 > FILE_BYTES_WRITTEN=3859160973 > FILE_READ_OPS=0 > FILE_LARGE_READ_OPS=0 > FILE_WRITE_OPS=0 > HDFS_BYTES_READ=2619256965 > HDFS_BYTES_WRITTEN=0 > HDFS_READ_OPS=21 > HDFS_LARGE_READ_OPS=0 > HDFS_WRITE_OPS=0 > org.apache.tez.common.counters.TaskCounter > SPILLED_RECORDS=45455352 > NUM_SHUFFLED_INPUTS=3 > NUM_FAILED_SHUFFLE_INPUTS=0 > GC_TIME_MILLIS=12756 > CPU_MILLISECONDS=1366940 > PHYSICAL_MEMORY_BYTES=15069728768 > VIRTUAL_MEMORY_BYTES=37774110720 > COMMITTED_HEAP_BYTES=18567659520 > INPUT_RECORDS_PROCESSED=36917387 > OUTPUT_RECORDS=83870869 > OUTPUT_LARGE_RECORDS=0 > OUTPUT_BYTES=8898320232 > OUTPUT_BYTES_WITH_OVERHEAD=8241464266 > OUTPUT_BYTES_PHYSICAL=3671993306 > ADDITIONAL_SPILLS_BYTES_WRITTEN=62138713 > ADDITIONAL_SPILLS_BYTES_READ=187158971 > ADDITIONAL_SPILL_COUNT=18 > SHUFFLE_BYTES=233406 > SHUFFLE_BYTES_DECOMPRESSED=493440 > SHUFFLE_BYTES_TO_MEM=233406 > SHUFFLE_BYTES_TO_DISK=0 > SHUFFLE_BYTES_DISK_DIRECT=0 > SHUFFLE_PHASE_TIME=689 > FIRST_EVENT_RECEIVED=228 > LAST_EVENT_RECEIVED=235 > org.apache.hadoop.mapreduce.TaskCounter > COMBINE_INPUT_RECORDS=20829242 > COMBINE_OUTPUT_RECORDS=23476241 > 2015-03-27 08:49:07,439 [PigTezLauncher-0] INFO > org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is > deprecated. Instead, use fs.defaultFS > 2015-03-27 08:49:07,597 [PigTezLauncher-0] INFO > org.apache.pig.tools.pigstats.JobStats - using output size reader: > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.FileBasedOutputSizeReader > 2015-03-27 08:49:08,220 [main] INFO > org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script Statistics: > > HadoopVersion: 2.5.0-cdh5.3.2 > PigVersion: 0.15.0-SNAPSHOT > TezVersion: 0.6.0 > UserId: root > FileName: foo.pig > StartedAt: 2015-03-27 08:47:40 > FinishedAt: 2015-03-27 08:49:08 > Features: HASH_JOIN,GROUP_BY,FILTER,UNION > > Failed! > > DAG PigLatin:foo.pig-0_scope-0: > ApplicationId: job_1427151599024_0195 > TotalLaunchedTasks: 51 > FileBytesRead: 187910630 > FileBytesWritten: 3859160973 > HdfsBytesRead: 2619256965 > HdfsBytesWritten: 0 > > Input(s): > Failed to read data from "pig.XXXX" > Successfully read 1000 records (171668 bytes) from: "pig.YYYY_sample" > Successfully read 13438146 records (483689478 bytes) from: "pig.ZZZZ" > > Output(s): > Failed to produce result in "hdfs:// > XXXX.YYYY.com:8020/tmp/temp-498428122/tmp-576253744" > > 2015-03-27 08:49:08,302 [main] ERROR org.apache.pig.tools.grunt.Grunt - > ERROR 1066: Unable to open iterator for alias cntallclaims_patient > Details at logfile: /root/PIG/pig_1427460455726.log > 2015-03-27 08:49:08,322 [main] INFO org.apache.pig.Main - Pig script > completed in 1 minute, 32 seconds and 719 milliseconds (92719 ms) > 2015-03-27 08:49:08,344 [main] INFO > org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting > down thread pool > 2015-03-27 08:49:08,394 [Thread-37] INFO > org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager - > Shutting down Tez session org.apache.tez.client.TezClient@37bdce24 > 2015-03-27 08:49:08,395 [Thread-37] INFO org.apache.tez.client.TezClient - > Shutting down Tez Session, sessionName=PigLatin:foo.pig, > applicationId=application_1427151599024_0195 > [root@ludwig PIG]# >
