Truong Duc Kien created FLINK-9458:
--------------------------------------
Summary: Unable to recover from job failure on YARN with NPE
Key: FLINK-9458
URL: https://issues.apache.org/jira/browse/FLINK-9458
Project: Flink
Issue Type: Bug
Affects Versions: 1.5.0
Environment: Ambari HDP 2.6.3
Hadoop 2.7.3
Job configuration:
120 Task Managers x 1 slots
Reporter: Truong Duc Kien
After upgrading our job to Flink 1.5, they are unable to recover from failure
with the following exception appears repeatedly
{noformat}
2018-05-29 04:56:06,086 [ jobmanager-future-thread-36] INFO
o.a.f.r.e.ExecutionGraph Try to restart or fail the job xxx
(23d9e87bf43ce163ff7db8afb062fb1d) if no longer possible. 2018-05-29
04:56:06,086 [ jobmanager-future-thread-36] INFO o.a.f.r.e.ExecutionGraph Job
xxx (23d9e87bf43ce163ff7db8afb062fb1d) switched from state RESTARTING to
RESTARTING. 2018-05-29 04:56:06,086 [ jobmanager-future-thread-36] INFO
o.a.f.r.e.ExecutionGraph Restarting the job xxx
(23d9e87bf43ce163ff7db8afb062fb1d). 2018-05-29 04:57:06,086 [
jobmanager-future-thread-36] WARN o.a.f.r.e.ExecutionGraph Failed to restart
the job. java.lang.NullPointerException at
org.apache.flink.runtime.jobmanager.scheduler.CoLocationConstraint.isAssignedAndAlive(CoLocationConstraint.java:104)
at
org.apache.flink.runtime.jobmanager.scheduler.CoLocationGroup.resetConstraints(CoLocationGroup.java:119)
at
org.apache.flink.runtime.executiongraph.ExecutionGraph.restart(ExecutionGraph.java:1247)
at
org.apache.flink.runtime.executiongraph.restart.ExecutionGraphRestartCallback.triggerFullRecovery(ExecutionGraphRestartCallback.java:59)
at
org.apache.flink.runtime.executiongraph.restart.FixedDelayRestartStrategy$1.run(FixedDelayRestartStrategy.java:68)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
