[
https://issues.apache.org/jira/browse/PIG-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17814970#comment-17814970
]
Koji Noguchi commented on PIG-5446:
-----------------------------------
From
{code:title=testProgressReportingWithStatusMessage}
cluster.setProperty(MRConfiguration.TASK_TIMEOUT, "10000");
...
pig.registerQuery("A = load 'a.txt' as (f1:chararray);");
pig.registerQuery("B = foreach A generate
org.apache.pig.test.utils.ReportingUDF();");
{code}
{code:title=ReportingUDF()}
public class ReportingUDF extends EvalFunc<Integer> {
@Override
public Integer exec(Tuple input) throws IOException {
try {
Thread.sleep(7500);
PigStatusReporter reporter = PigStatusReporter.getInstance();
reporter.progress();
Thread.sleep(7500);
} catch (InterruptedException e) {
}
return 100;
}
}
{code}
So basically, even though pig is calling "reporter.progress()" after 7.5
seconds, Tez task is failing with "Attempt failed because it appears to make no
progress for 10000ms".
> Tez TestPigProgressReporting.testProgressReportingWithStatusMessage failing
> ---------------------------------------------------------------------------
>
> Key: PIG-5446
> URL: https://issues.apache.org/jira/browse/PIG-5446
> Project: Pig
> Issue Type: Bug
> Components: tez
> Reporter: Koji Noguchi
> Assignee: Koji Noguchi
> Priority: Major
>
> {noformat}
> Unable to open iterator for alias B. Backend error : Vertex failed,
> vertexName=scope-4, vertexId=vertex_1707216362777_0001_1_00,
> diagnostics=[Task failed, taskId=task_1707216362777_0001_1_00_000000,
> diagnostics=[TaskAttempt 0 failed, info=[Attempt failed because it appears to
> make no progress for 10000ms], TaskAttempt 1 failed, info=[Attempt failed
> because it appears to make no progress for 10000ms]], Vertex did not succeed
> due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex
> vertex_1707216362777_0001_1_00 [scope-4] killed/failed due
> to:OWN_TASK_FAILURE] DAG did not succeed due to VERTEX_FAILURE.
> failedVertices:1 killedVertices:0
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias B. Backend error : Vertex failed, vertexName=scope-4,
> vertexId=vertex_1707216362777_0001_1_00, diagnostics=[Task failed,
> taskId=task_1707216362777_0001_1_00_000000, diagnostics=[TaskAttempt 0
> failed, info=[Attempt failed because it appears to make no progress for
> 10000ms], TaskAttempt 1 failed, info=[Attempt failed because it appears to
> make no progress for 10000ms]], Vertex did not succeed due to
> OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex
> vertex_1707216362777_0001_1_00 [scope-4] killed/failed due
> to:OWN_TASK_FAILURE]
> DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
> at org.apache.pig.PigServer.openIterator(PigServer.java:1014)
> at
> org.apache.pig.test.TestPigProgressReporting.testProgressReportingWithStatusMessage(TestPigProgressReporting.java:58)
> Caused by: org.apache.tez.dag.api.TezException: Vertex failed,
> vertexName=scope-4, vertexId=vertex_1707216362777_0001_1_00,
> diagnostics=[Task failed, taskId=task_1707216362777_0001_1_00_000000,
> diagnostics=[TaskAttempt 0 failed, info=[Attempt failed because it appears to
> make no progress for 10000ms], TaskAttempt 1 failed, info=[Attempt failed
> because it appears to make no progress for 10000ms]], Vertex did not succeed
> due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex
> vertex_1707216362777_0001_1_00 [scope-4] killed/failed due
> to:OWN_TASK_FAILURE]
> DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
> at
> org.apache.pig.tools.pigstats.tez.TezPigScriptStats.accumulateStats(TezPigScriptStats.java:204)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:243)
> at
> org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:212)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 45.647 {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
